ciszko.com

Using Firebase internal API before it was cool

Back in the day, one of the main struggles that I faced in my mobile automation project was app versioning. Manual testers used Testflight for iOS, which by the way is very convenient, but only for manual testing. Using it for the automation would be a nightmare, opening another app before the test session start, means that a lot of things can go wrong. For Android on the other hand, the apps were uploaded to multiple services like artifactory, firebase and circleCI. Artifactory was pretty easy to access as all you needed was just an API key. And so we stayed in this limbo for quite a while, android was downloaded from artifactory, iOS was being built from a scratch with some convoluted xcodebuild commands, until the day came when we discovered that the iOS apps were being uploaded to Firebase. A huge relief, it is Firebase, Google surely has made an API to download the app, surely right?

I played with Firebase for a while and found out that from the browser perspective you can download the app just fine.

Downloading a test app from firebase

Then came the Firebase REST API documentation but as it turns out, 3 years ago (2022) there was no way of downloading the app. The only thing that was doable was listing the releases and there was no information about the download URL or a binary. (now it is updated and each release in a list contains binaryDownloadUri that points to a link). So close, yet so far, but I couldn't lie my weapons down, maybe it is possible to mimic a browser behavior.

So the digging has started. By investigating the page code I saw that the "Download" button redirects to a URL that follows the pattern:

https://firebaseappdistribution.googleapis.com/app-binary-downloads/firebase-app-distro/app-binaries/[PROJECT_ID]/[APP_ID]/e4faf167-0e3d-4b1a-82dd-44811c0b5e43.apk?token=[TOKEN]

PROJECT_ID and APP_ID are unique identifiers given by Firebase when creating a project (these two will be static values). The part after APP_ID is the app identifier which can be actually retrieved from the API, the one that is documented. Using python it was pretty easy to setup, all I needed was the google.auth.transport.requests.AuthorizedSession object, which needed valid Google service account credentials. Fortunately there is a neat tutorial on how to get these, here. With that, I could call the list endpoint that returned all the versions of the app, where each version had a structure:

{
  "name": string,
  "releaseNotes": {
    object (ReleaseNotes)
  },
  "displayVersion": string,
  "buildVersion": string,
  "createTime": string
}

The name follows a pattern: projects/{projectNumber}/apps/{appId}/releases/{releaseId} and the releaseId part is what comes after APP_ID in the previous request. Unfortunately calling it was not working as I couldn't figure out, how TOKEN was being generated and copying it over was also a no go.

Upon further network traffic investigation, pressing the "Download" button actually triggered 2 requests. The first one is to get the download URL and the second one is just calling the URL returned by the previous request. First of two requests has the address like

https://firebaseappdistribution-pa.clients6.google.com/v1/{app_name}:getLatestBinary?alt=json&key={api_key}

The app_name looked like the name field from authenticated session when getting all app releases. The api_key I just copied over from the browser. Having a full URL constructed I tried sending GET but with no luck. All I got was a 403 Forbidden. That is something, seems like I was missing the headers. I copied all the headers and tried once again, still 403, so the next place that I had to look into were cookies.

When it comes to the cookies, things are slightly more complicated, the request requires following cookies: APISID, HSID, SAPISID, SID, SAPISIDHASH and `SSID. I copied them from the first request, tried to send a request and it worked! But I couldn't enjoy the victory for a long time as by the time I came back from a lunch it stopped working. Once again I fixed it by copying over the new cookies. Seemed like they were being generated, but based on what?

Searching the Internet with the names of the cookies I stumbled upon this Stackoverflow post that tried to reverse engineere the Google+ button and holy smokes they did it! The SAPISIDHASH cookie uses the SAPISID value, current epoch time and the origin value from the headers. Translating their solution into python looks like this:

def calculate_sapisid_hash(cookies):
    """Calculates SAPISIDHASH based on cookies. Required in authorization to download apk from firebase"""
    epoch = int(time())
    sha = sha1(' '.join([str(int(epoch)), cookies['SAPISID'], 'https://console.firebase.google.com']).encode())
    return f'SAPISIDHASH {int(epoch)}_{sha.hexdigest()}'

Finally by adding SAPISIDHASH to the cookies, we can send GET request on the first URL and get a valid response with "fileURL" and "fileSize" in it. Having "fileURL" we can save the final file with requests.get, wget or curl as there is no additional authorization needed.

One final touch was to shrink down the necessary headers that came down to:

{
    'origin': 'https://console.firebase.google.com',
    'referer': 'https://console.firebase.google.com/',
    'x-goog-authuser': '<has to be copied from the first request from browser>'
}

and voila! After a year and a half Google added binaryDownloadUri to the attributes of the release and this whole magic was not needed anymore but if there is one thing that I carried out of it, is that if something is accessible through the UI it surely can be accessed with the code as well. All you need is a little bit of stuborness and knowing where to look for clues.