Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add wrappers for native XCTest video recorder #858

Merged
merged 5 commits into from
Mar 7, 2024
Merged

Conversation

mykola-mokhnach
Copy link

It is a resurrection of #724 with some major changes.

I was able to figure out that videos are actually available, but cannot be fetched directly from WDA. Instead they are stored into the testmanagerd's container. On Simulators the path looks like $HOME/Library/Developer/CoreSimulator/Devices/F8E1968A-8443-4A9A-AB86-27C54C36A2F6/data/Containers/Data/InternalDaemon/4E3FE8DF-AD0A-41DA-B6EC-C35E5798C219/Attachments/A044DAF7-4A58-4CD5-95C3-29B4FE80C377

Where $HOME/Library/Developer/CoreSimulator/Devices/F8E1968A-8443-4A9A-AB86-27C54C36A2F6/ is the common Simulators containers root, InternalDaemon/4E3FE8DF-AD0A-41DA-B6EC-C35E5798C219/Attachments is the com.apple.testmanagerd 's domain and the A044DAF7-4A58-4CD5-95C3-29B4FE80C377 is the attachment UUID, which is uniquely assigned to each screen recording future upon startup.

With the above knowledge we can already fetch these videos from simulators. Need to only figure out how to get the access to this container for real devices...

@KazuCocoa

This comment was marked as resolved.

@mykola-mokhnach

This comment was marked as resolved.

@KazuCocoa

This comment was marked as outdated.

@KazuCocoa

This comment was marked as resolved.

@KazuCocoa
Copy link
Member

KazuCocoa commented Mar 5, 2024

% xcrun devicectl device info files --device=00008020-000E5CDA0A23002E --domain-type=appDataContainer --username mobile --domain-identifier=com.apple.testmanagerd  --subdirectory=Attachments
00:05:12  Enabling developer disk image services.
00:05:12  Acquired usage assertion.
3 files:
Name
------------------------------------
08B61FA8-80CC-47F1-A214-64988F827E7A
0C177732-2609-4117-B464-AE77D9E1B0BE
EAD01F1D-C247-4036-B577-7FFB9A03B5CC

@KazuCocoa
Copy link
Member

KazuCocoa commented Mar 5, 2024

% xcrun devicectl device copy from --device=00008020-000E5CDA0A23002E --domain-type=appDataContainer --user mobile --domain-identifier=com.apple.testmanagerd  --source=Attachments/08B61FA8-80CC-47F1-A214-64988F827E7A  --destination=./file.mp4
00:34:19  Acquired tunnel connection to device.
00:34:19  Enabling developer disk image services.
00:34:19  Acquired usage assertion.

succeeded in getting a file to the local file.mp4.

hm, no method to delete stored file via devicectl...?

@mykola-mokhnach
Copy link
Author

% xcrun devicectl device copy from --device=00008020-000E5CDA0A23002E --domain-type=appDataContainer --user mobile --domain-identifier=com.apple.testmanagerd  --source=Attachments/08B61FA8-80CC-47F1-A214-64988F827E7A  --destination=./file.mp4
00:34:19  Acquired tunnel connection to device.
00:34:19  Enabling developer disk image services.
00:34:19  Acquired usage assertion.

succeeded in getting a file to the local file.mp4.

hm, no method to delete stored file via devicectl...?

It's actually .mov rather than .mp4:

file 577EA9E0-4C93-4DE1-AE5E-8489A29590FA
577EA9E0-4C93-4DE1-AE5E-8489A29590FA: ISO Media, Apple QuickTime movie, Apple QuickTime (.MOV/QT)

Regarding file deletion, I've added a workaround https://github.com/appium/WebDriverAgent/pull/858/files#diff-f437ff5a0a8b42227214d5e8edf7bba1aa8ca97eb4a9632f8f95394996b19b7eR49, although I can only observe files are deleted after WDA quits completely (calling /wda/shutdown)

From native API perspective it is only possible to initiate file deletion via

@protocol XCTMessagingRole_AttachmentsDeleting
- (id)_IDE_deleteAttachmentsWithUUIDs:(NSSet *)arg1;
@end

protocol. This protocol is solely implemented by XCTMessagingChannel_IDEToDaemon channel, which means it could only be called via an "external" tunnel interface (which appium-ios-device does not support, same as we don't support images mounting).

@KazuCocoa
Copy link
Member

I confirmed when the WDA process ended with the shutdown endpoint, the created files were removed

@KazuCocoa
Copy link
Member

KazuCocoa commented Mar 6, 2024

i wonder if we should block this endpoint for real devices with a security flag to avoid accidental usage at least in appium-xcuitest layer

let me check if reset device (erase all data) could remove remained files

[update]
I have also confirmed erase all data removed all files as well created by this.
So it would be probably nice to leave comment or document in XCTest driver erase all data can remove these created files.

lgtm, overall

@mykola-mokhnach
Copy link
Author

@Dan-Maor Do you have anything to add?

@Dan-Maor
Copy link
Collaborator

Dan-Maor commented Mar 6, 2024

Sorry, it has been a hectic few days at the office so I wasn't able to look into it.

I would like to run some tests with my devices, however I'm working from home today and the device I have here at the moment failed to start recording with a "does not support screen recording" error (iOS 15.7).

I will be at the office tomorrow and will run some tests as soon as I can.

@mykola-mokhnach mykola-mokhnach marked this pull request as ready for review March 6, 2024 14:44
@mykola-mokhnach
Copy link
Author

I would like to run some tests with my devices, however I'm working from home today and the device I have here at the moment failed to start recording with a "does not support screen recording" error (iOS 15.7).

this is expected. The feature has been added by Apple only since Xcode 15/iOS 17

Copy link
Collaborator

@Dan-Maor Dan-Maor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me overall, however I did notice that files are actually not removed when stopping the test on my 17.4 device. Give that at this point devicectl does not allow file removal and the only way to remove these is a factory reset, I tend to agree with @KazuCocoa that allowing recording should be enabled by a flag to avoid accidental usage, at least until a way can be found to remove the recordings.

@mykola-mokhnach mykola-mokhnach merged commit 9728548 into master Mar 7, 2024
46 checks passed
@mykola-mokhnach mykola-mokhnach deleted the video_2 branch March 7, 2024 16:11
github-actions bot pushed a commit that referenced this pull request Mar 7, 2024
## [7.1.0](v7.0.6...v7.1.0) (2024-03-07)

### Features

* Add wrappers for native XCTest video recorder ([#858](#858)) ([9728548](9728548))

### Miscellaneous Chores

* bump typescript ([89880f5](89880f5))
Copy link

github-actions bot commented Mar 7, 2024

🎉 This PR is included in version 7.1.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@JumBe08
Copy link

JumBe08 commented Mar 8, 2024

The XCUITest Driver already had a method to record the screen, which would also make available a stream of the screen. This new feature is just about recording the screen or can it also be used to stream it in a better way?

@mykola-mokhnach
Copy link
Author

The XCUITest Driver already had a method to record the screen, which would also make available a stream of the screen. This new feature is just about recording the screen or can it also be used to stream it in a better way?

It cannot be used for streaming as we only get the recorded video at the very end when we call the stop API

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants