Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Searching for apps #12

Closed
baltpeter opened this issue May 21, 2024 · 10 comments
Closed

Searching for apps #12

baltpeter opened this issue May 21, 2024 · 10 comments
Labels
feature New feature or request research

Comments

@baltpeter
Copy link
Member

Just as in baltpeter/parse-play#9, we also need a search endpoint here.

@baltpeter baltpeter added feature New feature or request research labels May 21, 2024
@baltpeter
Copy link
Member Author

As usual with Apple, there are quite a few options as for which endpoint to use. Let's go through the ones I am aware of.

@baltpeter
Copy link
Member Author

First of all, as I already mentioned, there is the iTunes Search API, which is already well documented by Apple themselves and actually meant for public consumption.

Example:

https://itunes.apple.com/search?term=facebook&country=DE&media=software&lang=en_us&version=2

That returns (abbreviated):

{
    "resultCount": 50,
    "results": [
        {
            "supportedDevices": [
                "iPhone5s-iPhone5s",
                "iPadAir-iPadAir",
                "iPadAirCellular-iPadAirCellular",
                "iPadMiniRetina-iPadMiniRetina",
                "iPadMiniRetinaCellular-iPadMiniRetinaCellular",
                "iPhone6-iPhone6",
                "iPhone6Plus-iPhone6Plus",
                "iPadAir2-iPadAir2",
                "iPadAir2Cellular-iPadAir2Cellular",
                "iPadMini3-iPadMini3",
                "iPadMini3Cellular-iPadMini3Cellular",
                "iPodTouchSixthGen-iPodTouchSixthGen",
                "iPhone6s-iPhone6s",
                "iPhone6sPlus-iPhone6sPlus",
                "iPadMini4-iPadMini4",
                "iPadMini4Cellular-iPadMini4Cellular",
                "iPadPro-iPadPro",
                "iPadProCellular-iPadProCellular",
                "iPadPro97-iPadPro97",
                "iPadPro97Cellular-iPadPro97Cellular",
                "iPhoneSE-iPhoneSE",
                "iPhone7-iPhone7",
                "iPhone7Plus-iPhone7Plus",
                "iPad611-iPad611",
                "iPad612-iPad612",
                "iPad71-iPad71",
                "iPad72-iPad72",
                "iPad73-iPad73",
                "iPad74-iPad74",
                "iPhone8-iPhone8",
                "iPhone8Plus-iPhone8Plus",
                "iPhoneX-iPhoneX",
                "iPad75-iPad75",
                "iPad76-iPad76",
                "iPhoneXS-iPhoneXS",
                "iPhoneXSMax-iPhoneXSMax",
                "iPhoneXR-iPhoneXR",
                "iPad812-iPad812",
                "iPad834-iPad834",
                "iPad856-iPad856",
                "iPad878-iPad878",
                "iPadMini5-iPadMini5",
                "iPadMini5Cellular-iPadMini5Cellular",
                "iPadAir3-iPadAir3",
                "iPadAir3Cellular-iPadAir3Cellular",
                "iPodTouchSeventhGen-iPodTouchSeventhGen",
                "iPhone11-iPhone11",
                "iPhone11Pro-iPhone11Pro",
                "iPadSeventhGen-iPadSeventhGen",
                "iPadSeventhGenCellular-iPadSeventhGenCellular",
                "iPhone11ProMax-iPhone11ProMax",
                "iPhoneSESecondGen-iPhoneSESecondGen",
                "iPadProSecondGen-iPadProSecondGen",
                "iPadProSecondGenCellular-iPadProSecondGenCellular",
                "iPadProFourthGen-iPadProFourthGen",
                "iPadProFourthGenCellular-iPadProFourthGenCellular",
                "iPhone12Mini-iPhone12Mini",
                "iPhone12-iPhone12",
                "iPhone12Pro-iPhone12Pro",
                "iPhone12ProMax-iPhone12ProMax",
                "iPadAir4-iPadAir4",
                "iPadAir4Cellular-iPadAir4Cellular",
                "iPadEighthGen-iPadEighthGen",
                "iPadEighthGenCellular-iPadEighthGenCellular",
                "iPadProThirdGen-iPadProThirdGen",
                "iPadProThirdGenCellular-iPadProThirdGenCellular",
                "iPadProFifthGen-iPadProFifthGen",
                "iPadProFifthGenCellular-iPadProFifthGenCellular",
                "iPhone13Pro-iPhone13Pro",
                "iPhone13ProMax-iPhone13ProMax",
                "iPhone13Mini-iPhone13Mini",
                "iPhone13-iPhone13",
                "iPadMiniSixthGen-iPadMiniSixthGen",
                "iPadMiniSixthGenCellular-iPadMiniSixthGenCellular",
                "iPadNinthGen-iPadNinthGen",
                "iPadNinthGenCellular-iPadNinthGenCellular",
                "iPhoneSEThirdGen-iPhoneSEThirdGen",
                "iPadAirFifthGen-iPadAirFifthGen",
                "iPadAirFifthGenCellular-iPadAirFifthGenCellular",
                "iPhone14-iPhone14",
                "iPhone14Plus-iPhone14Plus",
                "iPhone14Pro-iPhone14Pro",
                "iPhone14ProMax-iPhone14ProMax",
                "iPadTenthGen-iPadTenthGen",
                "iPadTenthGenCellular-iPadTenthGenCellular",
                "iPadPro11FourthGen-iPadPro11FourthGen",
                "iPadPro11FourthGenCellular-iPadPro11FourthGenCellular",
                "iPadProSixthGen-iPadProSixthGen",
                "iPadProSixthGenCellular-iPadProSixthGenCellular",
                "iPhone15-iPhone15",
                "iPhone15Plus-iPhone15Plus",
                "iPhone15Pro-iPhone15Pro",
                "iPhone15ProMax-iPhone15ProMax"
            ],
            "advisories": [
                "Infrequent/Mild Mature/Suggestive Themes",
                "Infrequent/Mild Profanity or Crude Humour",
                "Infrequent/Mild Alcohol, Tobacco, or Drug Use or References",
                "Infrequent/Mild Sexual Content and Nudity"
            ],
            "isGameCenterEnabled": false,
            "features": ["iosUniversal"],
            "screenshotUrls": [
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource221/v4/e4/bd/90/e4bd9024-b230-1d51-d7e3-3b2634a217a2/f254967c-6ef9-4cc6-b4bf-e89cf016337c_1-iOS-5.5-Home.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource211/v4/85/42/a9/8542a90d-7954-e9dd-4e71-e2c7f650f260/c795c58d-ec37-463c-9ab9-ba0b8bc00ddd_2-iOS-5.5-Reels.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource211/v4/88/78/95/88789523-7cbf-65af-f94f-5e11d194bf5e/0130e841-7bad-4f8c-83c9-61ea0bac7d7b_3-iOS-5.5-Stories.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource221/v4/43/d4/d5/43d4d5fa-4b24-2e6d-6da3-bb700d118ceb/9b761f68-4cd1-4c66-b5f2-76de4f58005f_4-iOS-5.5-Groups.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource221/v4/7f/18/c6/7f18c63a-e3e6-ee8a-eb46-6498ed726b9f/68c29824-a52c-4766-8aea-0a70c7a2f86f_5-iOS-5.5-Profile.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource211/v4/1a/5a/1a/1a5a1a76-7315-f75b-7a19-c94b7dfa651a/28ff37e9-d855-4e6e-a8b4-fb2d585bb9ca_6-iOS-5.5-Marketplace.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource221/v4/cd/eb/30/cdeb30f1-a30a-68ff-643e-97bf8de92ba4/691c88f4-f976-46bb-92e1-5f088e8aa47f_7-iOS-5.5-Share.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource211/v4/3d/d6/76/3dd676e3-454b-ce42-4cc1-17adc7ff0ef6/d49c0719-9500-4504-8a4c-3124b914bafa_8-iOS-5.5-Search_with_AI.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource211/v4/c3/c6/6e/c3c66e8f-1167-ce8d-e778-19cc91915f8e/ee650e7b-1eaa-4de6-81a2-0b35dac16446_9-iOS-5.5-Write_with_AI.png/392x696bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/PurpleSource221/v4/59/ff/94/59ff9419-4bd1-6c15-0c70-2d180e4d4598/89fe27f2-ff0a-4a61-a502-73c2bd05da89_10-iOS-5.5-Notifications.png/392x696bb.png"
            ],
            "ipadScreenshotUrls": [
                "https://is1-ssl.mzstatic.com/image/thumb/Purple211/v4/a4/24/09/a424093a-29ec-fe7f-eb01-dee1be5a4e7d/3d69f402-0570-4aa0-b4ae-3b186d3098f3_iPad_Pro_12.9_2nd_Gen-Home.png/576x768bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/Purple211/v4/15/88/04/158804cf-67d3-d322-43ba-64fca479a8c0/e3a393bd-7c74-4e48-9868-1e93adb97c24_iPad_Pro_12.9_2nd_Gen-Watch.png/576x768bb.png",
                "https://is1-ssl.mzstatic.com/image/thumb/Purple211/v4/15/e0/92/15e09217-9d0b-000a-8824-5442ed426528/88d21855-709c-4c4e-b871-f74053b24d6f_iPad_Pro_12.9_2nd_Gen-Self_Profile.png/576x768bb.png"
            ],
            "appletvScreenshotUrls": [],
            "artworkUrl60": "https://is1-ssl.mzstatic.com/image/thumb/Purple221/v4/d1/7b/48/d17b48bc-21c2-440a-e91a-a76898f5dfed/Icon-Production-0-0-1x_U007emarketing-0-7-0-85-220.png/60x60bb.jpg",
            "artworkUrl512": "https://is1-ssl.mzstatic.com/image/thumb/Purple221/v4/d1/7b/48/d17b48bc-21c2-440a-e91a-a76898f5dfed/Icon-Production-0-0-1x_U007emarketing-0-7-0-85-220.png/512x512bb.jpg",
            "artworkUrl100": "https://is1-ssl.mzstatic.com/image/thumb/Purple221/v4/d1/7b/48/d17b48bc-21c2-440a-e91a-a76898f5dfed/Icon-Production-0-0-1x_U007emarketing-0-7-0-85-220.png/100x100bb.jpg",
            "artistViewUrl": "https://apps.apple.com/de/developer/meta-platforms-inc/id284882218?l=en&uo=4",
            "kind": "software",
            "currentVersionReleaseDate": "2024-05-20T12:00:42Z",
            "releaseNotes": "We’ve updated the app to fix some crashes, make features load faster and support iOS 17!",
            "artistId": 284882218,
            "artistName": "Meta Platforms, Inc.",
            "genres": ["Social Networking"],
            "price": 0.0,
            "description": "Explore, connect and share your interests with real people. \n\nWhether you’re looking for a spark of inspiration with reels or want to dive deeper into something you already love with Marketplace or in groups, you can discover ideas, experiences and people that fuel your interests and help you make progress on the things that matter to you on Facebook.\n\nExplore and expand your interests:\n* Shop for affordable and uncommon stuff on Marketplace and take your hobbies to the next level\n* Personalize your Feed to see more of what you like, less of what you don’t\n* Watch reels for quick entertainment that sparks inspiration\n* Discover creators, small businesses and communities who can help you dive deeper into the things you care about\n\nConnect with people and communities:\n* Join groups to learn tips and tricks from real people who’ve been there, done that\n* Catch up with friends, family and influencers through Feed and stories\n\nShare your world:\n* Effortlessly create reels from trending templates, or let your creativity shine with a full suite of editing tools\n* Customize your profile to choose how you show up and who you share your posts with\n* Turn your hobby into a side hustle by becoming a creator or selling things on Marketplace\n* Celebrate everyday, candid moments with stories, which disappear in 24 hours",
            "sellerName": "Meta Platforms, Inc.",
            "trackId": 284882215,
            "trackName": "Facebook",
            "primaryGenreName": "Social Networking",
            "primaryGenreId": 6005,
            "isVppDeviceBasedLicensingEnabled": true,
            "currency": "EUR",
            "genreIds": ["6005"],
            "bundleId": "com.facebook.Facebook",
            "minimumOsVersion": "13.4",
            "averageUserRatingForCurrentVersion": 4.13501000000000029643842935911379754543304443359375,
            "averageUserRating": 4.13501000000000029643842935911379754543304443359375,
            "trackCensoredName": "Facebook",
            "languageCodesISO2A": [
                "AR",
                "HR",
                "CS",
                "DA",
                "NL",
                "EN",
                "FI",
                "FR",
                "DE",
                "EL",
                "HE",
                "HI",
                "HU",
                "ID",
                "IT",
                "JA",
                "KO",
                "MS",
                "NB",
                "PL",
                "PT",
                "RO",
                "RU",
                "ZH",
                "SK",
                "ES",
                "SV",
                "TH",
                "ZH",
                "TR",
                "UK",
                "VI"
            ],
            "fileSizeBytes": "343912448",
            "sellerUrl": "http://www.facebook.com/mobile",
            "formattedPrice": "Free",
            "contentAdvisoryRating": "12+",
            "userRatingCountForCurrentVersion": 592702,
            "trackViewUrl": "https://apps.apple.com/de/app/facebook/id284882215?l=en&uo=4",
            "trackContentRating": "12+",
            "releaseDate": "2019-02-05T08:00:00Z",
            "version": "464.0.0",
            "wrapperType": "software",
            "userRatingCount": 592702
        },
        // …
    ]
}

@baltpeter
Copy link
Member Author

I don't have a jailbroken iPhone ready and the App Store unfortunately uses cert pinning. I really don't feel like going through that effort right now.

I'll try iTunes on Windows instead.

@baltpeter
Copy link
Member Author

Here's what iTunes on Windows does:

image

And the old x-apple-store-front trick (#1) trick also works. If I set X-Apple-Store-Front: 143443,30, I get a nice JSON instead of the annoying HTML with a <script> tag that holds the data.

However, this endpoint is still quite annoying.

For one, it fetches the data for all media types (?) at once: iPhone apps, iPad apps, Apple Watch apps, iMessage apps (that's a thing? o.o), books, audiobooks, albums, songs, podcasts, podcast episodes, music videos, TV episodes, TV series

image

That's a lot of wasted data.

Additionally, the response only contains the metadata for a random-looking unordered assortment of entries:

image

For the rest, you only get "bubbles" with the IDs:

image

iTunes then fetches the metadata separately in batches:

image

@baltpeter
Copy link
Member Author

As I also mentioned before, there is also an internal API of App Store Marketing Tools:

https://tools.applemediaservices.com/api/apple-media/apps/US/search.json?types=apps&term=facebook&limit=25&l=en-US&platform=iphone&additionalPlatforms=iphone,mac,appletv,ipad,watch,web

The big advantage here is that this returns the same format as the get app details endpoint we're using:

image

That makes me gravitate quite heavily towards this endpoint.

@baltpeter
Copy link
Member Author

baltpeter commented May 21, 2024

The disadvantage is that Apple seems to have locked down the endpoint a bit since I last looked at it. It is now quite finicky with what it accepts. For example, using anything other than limit=25 will result in a 403 (even if I specify a lower limit).

@baltpeter
Copy link
Member Author

As I have also alluded to in the linked issue, this API also lives at https://amp-api-edge.apps.apple.com/v1/catalog/us/search, which as far as I understand it should be the main endpoint in fact (the applemediaservices.com one is just a proxy). For that one, you need a token as I explained in the other issue and so my guess is that the applemediaservices.com just has a very limited token.

Unfortunately, the token from the App Store web page (which we are using for the app details endpoint) doesn't appear to work with the search endpoint—I always get a 403…

@baltpeter
Copy link
Member Author

OK. Well, for the time being, I think we should just use the applemediaservices.com endpoint. The consistency between our functions seems like the strongest argument to me. And if we later find a better way to access the same endpoint, we can always switch.

@baltpeter baltpeter changed the title Searching Searching for apps May 21, 2024
baltpeter added a commit that referenced this issue May 21, 2024
@kkthxbye-code
Copy link

I don't have a jailbroken iPhone ready and the App Store unfortunately uses cert pinning. I really don't feel like going through that effort right now.

Not sure if useful, but here's a search request from an iPhone I did recently (truncated headers). I guess the interesting part is amp-api-search-edge.apps.apple.com - however like you discovered, it doesn't seem possible to use an apple.com token. I opted to use the official search API instead, though it is heavily rate limited.

:method: GET
:scheme: https
:path: /v1/catalog/dk/search?additionalPlatforms=appletv%2Cipad%2Cmac%2Cwatch&associate%5Bapps%5D=app-events&bubble%5Bsearch%5D=apps%2Cdevelopers%2Cgroupings%2Ceditorial-items%2Capp-bundles%2Cin-apps&extend=customArtwork%2CcustomScreenshotsByType%2CcustomScreenshotsByTypeForAd%2CcustomVideoPreviewsByType%2CcustomVideoPreviewsByTypeForAd%2CeditorialBadgeInfo%2CeditorialVideo%2CmessagesScreenshots%2CminimumOSVersion%2CremoteControllerRequirement%2CrequiredCapabilities%2CsupportsFunCamera&extend%5Bapp-events%5D=description%2CproductArtwork%2CproductVideo&extend%5Beditorial-items%5D=showLabelInSearch&include=apps%2Ctop-apps&include%5Beditorial-items%5D=marketing-items%2Cprimary-content&l=en-GB&limit%5Bads-result%5D=4&meta%5Bmarketing-items%5D=metrics&platform=iphone&term=teststring&with=guidedSearch
:authority: amp-api-search-edge.apps.apple.com
x-apple-tz: 7200
cookie: X-Dsid=21237191337
cookie: xt-src=b
cookie: pldfltcid=598ae...
cookie: vrep=CKPZ14...
cookie: tv-pldfltcid=598ae...
cookie: xp_ab=1#IJtoLUK...
cookie: xt-b-ts-21237191337=170...
cookie: mz_at_ssl-21237191337=AwU...
cookie: mz_at0-21237191337=AwQ...
cookie: wosid-lite=mYe...
cookie: fsas=AAA...
icloud-dsid: 21237191337
x-dsid: 21237197987
x-apple-store-front: 143458-2,29 t:apps3
x-apple-iad-request-data: AAAA...
user-agent: AppStore/3.0 iOS/16.7.4 model/iPhone10,6 hwp/t8015 build/20H240 (6; dt:162) AMS/1
x-apple-iad-env-name: AAAA...
x-apple-client-application: com.apple.AppStore
x-apple-i-timezone: CEST
x-apple-i-client-time: 2024-05-14T11:38:...
x-apple-app-store-client-request-id: 046...
authorization: Bearer ey...
accept-language: en-DK
x-apple-i-md-rinfo: 50...
x-apple-adsid: 001...
accept: */*
accept-encoding: br, gzip, deflate
x-apple-i-md-m: LK...
x-apple-i-locale: en_DK
x-apple-i-md: AAA...

@zner0L zner0L closed this as completed in b733096 Jun 11, 2024
@baltpeter
Copy link
Member Author

Thank you, @kkthxbye-code. That's helpful context!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request research
Projects
None yet
Development

No branches or pull requests

2 participants