Unable to run iOS App test targets when pendo installed via pod

[benjamin-es-hall]

We are having issues when trying to run our tests with pendo (v2.18.0) installed via cocoa pods. This manifests as the app/simulator failing to launch when running tests, both locally and over CI (Xcode Cloud in our case). When Pendo is installed via SPM the issue does not seem to arise.

The issue seems somewhat similar to this https://developer.apple.com/forums/thread/704899

Let me know if I can provide more information.

[MikePendo]

@benjamin-es-hall
The issue u have linked was as fault of Texture SDK that we use for drawing the UI and was resolved on our side XCTests using Texture/AsyncDisplayKit deadlock during initialisation when targeting iOS 15
Let me create a sample project to see if we can reproduce the current issue u r talking about.
Can you provide some additional logs about the issue u experience?
If you exclude Pendo can u run the tests?
Which IOS version u r running on?
what is the deployment target?
What Xcode command line tools u r using?
Can u please share the exact version of Pendo Sdk (including patch number)
Its kinda strange (that u mange to test with SPM and it doesnt work with cocoapods)

[benjamin-es-hall]

Thanks for the quick reply!

Let me create a sample project to see if we can reproduce the current issue u r talking about.

I just tried this and seemingly there was no issue, irrespective of pod or SPM installation. This has made me lean more towards a specific project setup but I'm yet to pinpoint what.

If you exclude Pendo can u run the tests?
Yes

Which IOS version u r running on?

Tested on 15.5 (sim, Xcode cloud)
16.0 & 16.1 (sim, beta Xcode)
16.0.2 (device)

what is the deployment target?
iOS 15.0

What Xcode command line tools u r using?
13.4.1

Can u please share the exact version of Pendo Sdk (including patch number)
2.18.0.6298

Its kinda strange (that u mange to test with SPM and it doesnt work with cocoapods)

Yeah this is peculiar.

Another test I tried was to add a different pod other than pendo and see if there is the same issue. My thinking was that this is the first new pod added to the project in a while so it may not actually be pendo related. However, it seems removing pendo and adding a different pod still allowed the tests to run. I don't think this rules out a project configuration issue, but is one more datapoint.

I'll update with any further details tomorrow

[MikePendo]

@benjamin-es-hall
So I have created a project with Xcode 13.4 (iOS 16 sim)
project
And I can run the test in Xcode.
Do u run the tests in Xcode or command line?
It get stuck during the build or during one of the tests?
if u run only single stand alone test will it stuck?
Maybe u could exclude all the data from the project u have the issue and share it with me so I will have a look at it (otherwise its really hard to say where the issue is coming from)

[benjamin-es-hall]

So here's a fun one. I tried playing around with other Pods and noticed an old version of Firebase Crashlytics/Analytics installed. I bumped it up to newest version and the tests seemingly started working.

I then went to my fresh project to see if adding the old versions of Firebase as well as Pendo caused the hanging when launching tests (all tests have been launched from Xcode rather than command line, apart from Xcode Cloud which I imagine launches from command line). And it did! 🤔

Here is the sample PodFile I used:

# Uncomment the next line to define a global platform for your project
platform :ios, '15.0'

target 'PendoTest' do
  # Comment the next line if you don't want to use dynamic frameworks
  use_frameworks!
  # Pods for PendoTest
  pod 'Firebase/Crashlytics','7.1.0'
  pod 'Firebase/Analytics', '7.1.0'
  pod 'Pendo', '2.18.0.6298'

  target 'PendoTestTests' do
    inherit! :search_paths
    # Pods for testing
  end

  target 'PendoTestUITests' do
    # Pods for testing
  end

  #Comment the next line if you don't want to use dynamic frameworks
  post_install do |installer|
    # Disable code coverage for all Pods and Pods Project
    installer.pods_project.targets.each do |target|
      target.build_configurations.each do |config|
       # Below only included as building to sim on M1 machine and Firebase pods listed above didn't include arm64 versions
        config.build_settings['EXCLUDED_ARCHS[sdk=iphonesimulator*]'] = 'arm64'
      end
    end
  end
end

If that also causes issues for you it seems a weird combination of something in Firebase in that version (newest 9.6.0 doesn't seem to trigger hang) and Pendo is causing launch hang in tests only 🤔

======== Answers to your comment:

Do u run the tests in Xcode or command line?

Xcode locally, I imagine command line with Xcode cloud

It get stuck during the build or during one of the tests?

After build and when launching app to test, eventually times out with no tests run and fails

if u run only single stand alone test will it stuck?

Yes

Maybe u could exclude all the data from the project u have the issue and share it with me so I will have a look at it (otherwise its really hard to say where the issue is coming from)

Let me know if PodFile above is enough, if not I'll prepare the sample

Thanks again!

[MikePendo]

@benjamin-es-hall
Thanks!
I managed to reproduce the issue (its kinda of deadlock that happens sometimes).
When u have such case just press the pause button in Xcode and see all the running threads, it looks like Pendo SDK and Firebase are using some common apple api which are not thread safe(first guess). (u will see the code get stuck on both reachability apis) I didnt manage to reproduce it with our open code only with compiled Pendo library.
U told that this issue happens only for this specific firebase SDK, is it possible for u to upgrade to newer firebase SDK so that wont block you (until we supply a fix)

[MikePendo]

@benjamin-es-hall
Again thanks for reporting the issue, we have resolved it. (cant really explain why the deadlock actually happened)
I will try to push it in our next release which is 2.18.2 (and coming next week)
I will close the thread as soon as we release the fix

[benjamin-es-hall]

Hi @MikePendo no problem! Thank you so much for looking into it so quickly and I'm glad we could find a repeatable way to trigger it! Also good tip about pausing the debugger, I didn't think of that!

We definitely can update our fire base dependency (it was on the todo list anyway), but I'm glad you have found a solution within Pendo as well!

Thanks again!

[MikePendo]

@benjamin-es-hall
we have merged it next 2.19 release which should be out next month (I hope)
I am closing the issue