1

Context

I have an application (let's call it App.app) that I'd like to automate using JXA. It doesn't show up in Script Editor's Dictionary. I guess that means that it does not expose an api to osascripts. But when I inspect App.app with Accessibility Inspector, all the UI elements that I want to manipulate by the automation are available. I can see the hierarchy and can click on them through Accessibility Inspector.

Question(s)

How can I use JXA or AppleScript to navigate the accessibility model of App.app so that I can click on the its UI elements? If impossible with JXA, what's the easiest way to accomplish that? Can you recommend a good source if information about JXA and exploiting applidations' exposed accesiblitiy options?

Research

I perused through "Similar questions" just now. Some, like this one seem to offer ways to explore towards solution. There is the Apple's description of accessibility model. But it focuses mostly on exposing accessibility options, not exploiting them.

Specifics

The app I'm automating is RubyMine/Intellij. Among other things, I want the automation to start a bunch of console tabs, run a certain command in each and rename each accordingluy.

Relevance

Finding the solution to this can empower anyone wanting to automate any application that exposes accessibility option

user2026920
  • 76
  • 1
    "If impossible, what's the easiest way to accomplish that?" This is asking a lot - literally to do the impossible. – Allan Apr 30 '20 at 23:36
  • Good point. I'll edit to clarify. – user2026920 Apr 30 '20 at 23:50
  • Accessibility is not the same as automation. If you are wanting something like a scripting dictionary, you will need to provide that yourself - see the Cocoa Scripting Guide. – red_menace May 01 '20 at 00:33
  • @red_menac accessibility in theory provides a way for assistive apps to control UI of other applications. Why not use that for ui automation? – user2026920 May 01 '20 at 02:35
  • 1
    GUI scripting can work, but it is normally considered a means of last resort, since pretty much anything can change (variations in the current instance, objects and their hierarchy, names or descriptions, whether or not an object has the desired properties defined, etc), which will throw off a script. The best approach is for the application to provide access to what it supports, so a script can just ask it. – red_menace May 01 '20 at 13:11

0 Answers0