Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add an endpoint for keyboard input #797

Merged
merged 2 commits into from
Oct 26, 2023
Merged

feat: Add an endpoint for keyboard input #797

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7882d6e
feat: Add an endpoint for keyboard input
mykola-mokhnach Oct 25, 2023
c8540df
Add conditions
mykola-mokhnach Oct 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions WebDriverAgentLib/Commands/FBCustomCommands.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,9 @@ + (NSArray *)routes
[[FBRoute GET:@"/wda/device/location"] respondWithTarget:self action:@selector(handleGetLocation:)],
[[FBRoute GET:@"/wda/device/location"].withoutSession respondWithTarget:self action:@selector(handleGetLocation:)],
#if !TARGET_OS_TV // tvOS does not provide relevant APIs
#if __clang_major__ >= 15
[[FBRoute POST:@"/wda/element/:uuid/keyboardInput"] respondWithTarget:self action:@selector(handleKeyboardInput:)],
#endif
[[FBRoute GET:@"/wda/simulatedLocation"] respondWithTarget:self action:@selector(handleGetSimulatedLocation:)],
[[FBRoute GET:@"/wda/simulatedLocation"].withoutSession respondWithTarget:self action:@selector(handleGetSimulatedLocation:)],
[[FBRoute POST:@"/wda/simulatedLocation"] respondWithTarget:self action:@selector(handleSetSimulatedLocation:)],
Expand Down Expand Up @@ -543,6 +546,48 @@ + (NSString *)timeZone
}
return FBResponseWithOK();
}

#if __clang_major__ >= 15
Copy link
Member

@KazuCocoa KazuCocoa Oct 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or add - (void)typeKey:(id)arg1 modifierFlags:(unsigned long long)arg2; in https://github.com/appium/WebDriverAgent/blob/master/PrivateHeaders/XCTest/XCUIElement.h and check with selector as we do usually?

On my local Xcode 14 only addressed this method, typeKey, but others such as XCUIKeyboardKeyDelete existed

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to focus on current OS support as Apple updates its stuff very fast.
Although feel free to test it locally and create a PR if needed to add the support of older versions.

Copy link
Member

@KazuCocoa KazuCocoa Oct 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, i'm interested in making wda package work for both iOS 17 and 16 real devices with iOS 17's way (built with Xcode 15), so no embedded testing frameworks from .app. I guessed at least a selector might be needed for iOS 16 to not crash the process with the endpoint call. I'll check for it, then create a pr to care about the case

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just chiming in here, I'm interested in IOS 16 as well, as there currently isn't any "nice" way of mounting developer image on IOS 17 on Linux.

Otherwise, I understand wanting to support the latest, with the fast movement.

+ (id<FBResponsePayload>)handleKeyboardInput:(FBRouteRequest *)request
{
FBElementCache *elementCache = request.session.elementCache;
BOOL hasElement = nil != request.parameters[@"uuid"];
XCUIElement *destination = hasElement
? [elementCache elementForUUID:(NSString *)request.parameters[@"uuid"]]
: request.session.activeApplication;
id keys = request.arguments[@"keys"];
if (![keys isKindOfClass:NSArray.class]) {
NSString *message = @"The 'keys' argument must be an array";
return FBResponseWithStatus([FBCommandStatus invalidArgumentErrorWithMessage:message
traceback:nil]);
}
for (id item in (NSArray *)keys) {
if ([item isKindOfClass:NSString.class]) {
NSString *keyValue = [FBKeyboard keyValueForName:item] ?: item;
[destination typeKey:keyValue modifierFlags:XCUIKeyModifierNone];
} else if ([item isKindOfClass:NSDictionary.class]) {
id key = [(NSDictionary *)item objectForKey:@"key"];
if (![key isKindOfClass:NSString.class]) {
NSString *message = [NSString stringWithFormat:@"All dictionaries of 'keys' array must have the 'key' item of type string. Got '%@' instead in the item %@", key, item];
return FBResponseWithStatus([FBCommandStatus invalidArgumentErrorWithMessage:message
traceback:nil]);
}
id modifiers = [(NSDictionary *)item objectForKey:@"modifierFlags"];
NSUInteger modifierFlags = XCUIKeyModifierNone;
if ([modifiers isKindOfClass:NSNumber.class]) {
modifierFlags = [(NSNumber *)modifiers unsignedIntValue];
}
NSString *keyValue = [FBKeyboard keyValueForName:item] ?: key;
[destination typeKey:keyValue modifierFlags:modifierFlags];
} else {
NSString *message = @"All items of the 'keys' array must be either dictionaries or strings";
return FBResponseWithStatus([FBCommandStatus invalidArgumentErrorWithMessage:message
traceback:nil]);
}
}
return FBResponseWithOK();
}
#endif
#endif

+ (id<FBResponsePayload>)handlePerformAccessibilityAudit:(FBRouteRequest *)request
Expand Down
10 changes: 10 additions & 0 deletions WebDriverAgentLib/Utilities/FBKeyboard.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,16 @@ NS_ASSUME_NONNULL_BEGIN

@interface FBKeyboard : NSObject

#if (!TARGET_OS_TV && __clang_major__ >= 15)
/**
Transforms key name to its string representation, which could be used with XCTest

@param name one of available keyboard key names defined in https://developer.apple.com/documentation/xctest/xcuikeyboardkey?language=objc
@return Either the key value or nil if no matches have been found
*/
+ (nullable NSString *)keyValueForName:(NSString *)name;
#endif

/**
Types a string into active element. There must be element with keyboard focus; otherwise an
error is raised.
Expand Down
65 changes: 65 additions & 0 deletions WebDriverAgentLib/Utilities/FBKeyboard.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,69 @@ + (BOOL)waitUntilVisibleForApplication:(XCUIApplication *)app timeout:(NSTimeInt
error:error];
}

#if (!TARGET_OS_TV && __clang_major__ >= 15)

+ (NSString *)keyValueForName:(NSString *)name
{
static dispatch_once_t onceKeys;
static NSDictionary<NSString *, NSString *> *keysMapping;
dispatch_once(&onceKeys, ^{
keysMapping = @{
@"XCUIKeyboardKeyDelete": XCUIKeyboardKeyDelete,
@"XCUIKeyboardKeyReturn": XCUIKeyboardKeyReturn,
@"XCUIKeyboardKeyEnter": XCUIKeyboardKeyEnter,
@"XCUIKeyboardKeyTab": XCUIKeyboardKeyTab,
@"XCUIKeyboardKeySpace": XCUIKeyboardKeySpace,
@"XCUIKeyboardKeyEscape": XCUIKeyboardKeyEscape,

@"XCUIKeyboardKeyUpArrow": XCUIKeyboardKeyUpArrow,
@"XCUIKeyboardKeyDownArrow": XCUIKeyboardKeyDownArrow,
@"XCUIKeyboardKeyLeftArrow": XCUIKeyboardKeyLeftArrow,
@"XCUIKeyboardKeyRightArrow": XCUIKeyboardKeyRightArrow,

@"XCUIKeyboardKeyF1": XCUIKeyboardKeyF1,
@"XCUIKeyboardKeyF2": XCUIKeyboardKeyF2,
@"XCUIKeyboardKeyF3": XCUIKeyboardKeyF3,
@"XCUIKeyboardKeyF4": XCUIKeyboardKeyF4,
@"XCUIKeyboardKeyF5": XCUIKeyboardKeyF5,
@"XCUIKeyboardKeyF6": XCUIKeyboardKeyF6,
@"XCUIKeyboardKeyF7": XCUIKeyboardKeyF7,
@"XCUIKeyboardKeyF8": XCUIKeyboardKeyF8,
@"XCUIKeyboardKeyF9": XCUIKeyboardKeyF9,
@"XCUIKeyboardKeyF10": XCUIKeyboardKeyF10,
@"XCUIKeyboardKeyF11": XCUIKeyboardKeyF11,
@"XCUIKeyboardKeyF12": XCUIKeyboardKeyF12,
@"XCUIKeyboardKeyF13": XCUIKeyboardKeyF13,
@"XCUIKeyboardKeyF14": XCUIKeyboardKeyF14,
@"XCUIKeyboardKeyF15": XCUIKeyboardKeyF15,
@"XCUIKeyboardKeyF16": XCUIKeyboardKeyF16,
@"XCUIKeyboardKeyF17": XCUIKeyboardKeyF17,
@"XCUIKeyboardKeyF18": XCUIKeyboardKeyF18,
@"XCUIKeyboardKeyF19": XCUIKeyboardKeyF19,

@"XCUIKeyboardKeyForwardDelete": XCUIKeyboardKeyForwardDelete,
@"XCUIKeyboardKeyHome": XCUIKeyboardKeyHome,
@"XCUIKeyboardKeyEnd": XCUIKeyboardKeyEnd,
@"XCUIKeyboardKeyPageUp": XCUIKeyboardKeyPageUp,
@"XCUIKeyboardKeyPageDown": XCUIKeyboardKeyPageDown,
@"XCUIKeyboardKeyClear": XCUIKeyboardKeyClear,
@"XCUIKeyboardKeyHelp": XCUIKeyboardKeyHelp,

@"XCUIKeyboardKeyCapsLock": XCUIKeyboardKeyCapsLock,
@"XCUIKeyboardKeyShift": XCUIKeyboardKeyShift,
@"XCUIKeyboardKeyControl": XCUIKeyboardKeyControl,
@"XCUIKeyboardKeyOption": XCUIKeyboardKeyOption,
@"XCUIKeyboardKeyCommand": XCUIKeyboardKeyCommand,
@"XCUIKeyboardKeyRightShift": XCUIKeyboardKeyRightShift,
@"XCUIKeyboardKeyRightControl": XCUIKeyboardKeyRightControl,
@"XCUIKeyboardKeyRightOption": XCUIKeyboardKeyRightOption,
@"XCUIKeyboardKeyRightCommand": XCUIKeyboardKeyRightCommand,
@"XCUIKeyboardKeySecondaryFn": XCUIKeyboardKeySecondaryFn
};
});
return keysMapping[name];
}

#endif

@end
Loading