Element Attributes and Actions

This document provides an overview of some methods in the hs.axuielement module for examining and manipulating accessibility elements.

This is just an informal overview showing some of the ways you can examine and manipulate elements. For full documentation of any function or method described here, you should consut the official documentation.

Examining an element

For the purposes of this first example, I selected the minimize button of an open Safari window. To select this element if you wish to follow along:

Make sure that you have a Safari window open and visible, and then open the Hammerspoon console.
Move the mouse pointer over the minimize button of the Safari window, but do not click on it -- just hover over it.
In the Hammerspoon console, type the following: b = hs.axuielement.systemElementAtPosition(hs.mouse.getAbsolutePosition())

Once you have an element object, you can generate a list of its attributes with the hs.axuielement:attributeNames method:

> hs.inspect(b:attributeNames())
{ "AXEnabled", "AXParent", "AXSize", "AXFocused", "AXRole", "AXTopLevelUIElement", "AXAuditIssues", "AXHelp", "AXPosition", "AXTitle", "AXWindow", "AXRoleDescription", "AXSubrole", "AXFrame" }

Any actions that can be performed on it can be queried with the hs.axuielement:actionNames method:

> hs.inspect(b:actionNames())
{ "AXPress" }

And any parameteried attributes supported by the element can be querried with the hs.axuielement:parameterizedAttributeNames method:

> hs.inspect(b:parameterizedAttributeNames())
{ "AXReplaceRangeWithText" }

(Parameterized attributes are attributes for which the returned value is partially determined by arguments you pass to the appropriate function when querying them -- they are beyond the scope of this document but will be explored more fully in a document which will be added to the Wiki at a later time. For now, they are included only for completeness.)

Querying Attribute Values for an Element

To query the value of a specific attribute, you can use the hs.axuielement:attributeValue method:

> b:attributeValue("AXRoleDescription")
minimize button

You can also use a shortcut of specifying the attribute name as if it were a field of the object:

> b.AXRoleDescription
minimize button

Functionally these are identical -- the shortcut is just for convienence.

There is also a shortcut to poll all attribute values at once, rather than individually. It should be noted that if you are only interested in one or two attribute values, this method is a little slower, but is still faster than querying all of them individually yourself:

> hs.inspect(b:allAttributeValues())
{
  AXAuditIssues = {},
  AXEnabled = true,
  AXFocused = false,
  AXFrame = {
    h = 16.0,
    w = 14.0,
    x = 392.0,
    y = 105.0
  },
  AXParent = <userdata 1> -- hs.axuielement: AXWindow (0x606003961558),
  AXPosition = {
    x = 392.0,
    y = 105.0
  },
  AXRole = "AXButton",
  AXRoleDescription = "minimize button",
  AXSize = {
    h = 16.0,
    w = 14.0
  },
  AXSubrole = "AXMinimizeButton",
  AXTopLevelUIElement = <userdata 2> -- hs.axuielement: AXWindow (0x60600409ff78),
  AXWindow = <userdata 3> -- hs.axuielement: AXWindow (0x60600090bf38)
}

The observant may notice that hs.axuielement:attributeNames also lists AXHelp and AXTitle for this object, but they are not present in this summary -- that is because Lua treats the absence of a value as nil and within a table of key-value pairs, a value of nil means that the key is removed from the table.

AXUIElement objects, however, treat an attribute that belongs to an object but has no value as an error. To more closely match the Lua behaviors, hs.axuielement:allAttributeValues will normally supress attributes that return an error, but if we wish to include those, we can pass the optional argument true to the method:

> hs.inspect(b:allAttributeValues(true))
{
  AXAuditIssues = {},
  AXEnabled = true,
  AXFocused = false,
  AXFrame = {
    h = 16.0,
    w = 14.0,
    x = 392.0,
    y = 105.0
  },
  AXHelp = {
    _code = -25212,
    error = "Requested value does not exist"
  },
  AXParent = <userdata 1> -- hs.axuielement: AXWindow (0x606000e6fb18),
  AXPosition = {
    x = 392.0,
    y = 105.0
  },
  AXRole = "AXButton",
  AXRoleDescription = "minimize button",
  AXSize = {
    h = 16.0,
    w = 14.0
  },
  AXSubrole = "AXMinimizeButton",
  AXTitle = {
    _code = -25212,
    error = "Requested value does not exist"
  },
  AXTopLevelUIElement = <userdata 2> -- hs.axuielement: AXWindow (0x60600189ad38),
  AXWindow = <userdata 3> -- hs.axuielement: AXWindow (0x606000180178)
}

As you can see in this second report, the missing attributes appear with the error code and description included within a table for that purpose.

Performing Actions on an Element

Some elements allow actions to be performed on them. As noted for the button we have chosen to investigate, "AXPress" has been defined for this element. To trigger this action you can use the hs.axuielement:performAction method:

> b:performAction("AXPress")
hs.axuielement: AXButton (0x60600142d818)

Performing actions also has a shortcut -- you can preface the action name with "do" and call it as you would any other method on the object:

> b:doAXPress()
hs.axuielement: AXButton (0x60600142d818)

Changing an attribute value

Some attributes can have their value changed. What effect this has on the element is highly dependant upon the specific element and application, but can be useful. In the case of the button, all of it's attributes are read only -- they cannot be changed. You can check a specific attribute with the hs.axuielement:isAttributeSettable method:

> for i,v in ipairs(b:attributeNames()) do print(v, b:isAttributeSettable(v)) end
AXEnabled	false
AXParent	false
AXSize	false
AXFocused	false
AXRole	false
AXTopLevelUIElement	false
AXAuditIssues	false
AXHelp	false
AXPosition	false
AXTitle	false
AXWindow	false
AXRoleDescription	false
AXSubrole	false
AXFrame	false

Let's look at the window of the button, though -- for this element, we can get at it through the AXWindow or AXParent property. It should be noted that in this case, they are one in the same, but for some elements which are members of a table cell or other grouping structure for layout purposes, they may differ -- while you can traverse the path back through one or more AXParent objects to get at the actual window, when AXWindow is an available attribute, it can be considered a shortcut that may bypass zero or more interveneing ancestors.

> bw = b.AXWindow

> for i,v in ipairs(bw:attributeNames()) do print(v, bw:isAttributeSettable(v)) end
AXFocused	false
AXFullScreen	true
AXTitle	false
AXChildrenInNavigationOrder	nil	A system error occurred
AXAuditIssues	nil	A system error occurred
AXPosition	true
AXGrowArea	false
AXMinimizeButton	false
AXDocument	false
AXSections	true
AXCloseButton	false
AXMain	false
AXFullScreenButton	false
AXProxy	false
AXDefaultButton	false
AXMinimized	true
AXChildren	false
AXRole	false
AXParent	false
AXTitleUIElement	false
AXCancelButton	false
AXModal	false
AXSubrole	false
AXZoomButton	false
AXRoleDescription	false
AXSize	true
AXToolbarButton	false
AXIdentifier	false
AXFrame	nil	Attribute is not supported by target

Here we see that AXMinimized is a settable attribute. Assuming your window is still minimized after issuing the AXPress action through one of the methods issued above, we can verify that by checking its value:

> bw.AXMinimized
true

Because AXMinimized is a settable attribute, we can change its value to restore the minimized window with the hs.axuielement:setAttributeValue method:

> bw:setAttributeValue("AXMinimized", false)
hs.axuielement: AXWindow (0x6060023a81b8)

Like getting an attribute value, you can use a shortcut method by treating the attribute name as a field: bw.AXMinimized = false would perform the equivalent action as the method above. It should be noted that in this case, however, there is a slight difference between the approaches -- if hs.axuielement:setAttributeValue encounters an error while setting the attribute to its new value, it will return nil, errorMessageAsAString. Using the short cut does not allow us to return an error so the only way to verify the change is to query the attribute afterwards to confirm the change in value.

Attribute Validity

User interface elements within an application are created and destroyed all the time depending upon what is currently being displayed by the application. Some elements have a predicatable life time: the application object will exist as long as the application is running, window objects will exist as long as the window is open, standard menu items within then menubar will exist for the lifetime of the application or until programatically changed (updating recently opened files, for example).

Other elements, such as those referring to transitory elements within a dialog box or belonging to pop-up menus, will only exist as long as they are visible. Because of their transitory nature, it can sometimes be a challenge to use them, but for an exampe of accessing items in a pop-up menu, check out the dockStuff.lua example.

Within your programs, you can check to see if an element is still valid, which usually means that it is still visible within the application, you can use the hs.axuielement:isValid method:

> bw:isValid()
true

Once the window is closed, however, the element will no longer be valid:

> bw.AXCloseButton:doAXPress()
hs.axuielement: *element invalid* (0x6060047550b8)

> bw:isValid()
false

As you can see, this is also visible in the string representation of the element that is printed to the Hammerspoon console, but rather than trying to match this as a string (e.g. tostring(bw):match("%*element invalid%*")), it is easier just to use the hs.axuielement:isValid method.

Conclusion

This has just been an introduction to some of the things you can do with the hs.axuielement module. As noted elsewhere, this module is more of a tool kit than an end in itself, so you should expect to perform some experimentation to discover what is possible. Hopefully this document has given you some ideas on how you can begin.

For further information, you can consult the following: