Integrations

I received a unique request to fetch PDF copies of bills from one of our utility providers, Southern California Edison. SCE provides a means of sending billing data in CSV format, but the requester requires PDF copies for an OCR process. Since our campus spans multiple buildings and residences and bills deliver at different times of the month, this required logging in multiple times a month to download 20+ different bills.

This seemed like a fun challenge that could result in some real time savings for the requester, so I got the credentials and began poking around in dev tools.

I began by filtering on Fetch/XHR requests and started linking them together. I quickly realized I had started dev tools too late. A session with the identity provider (Okta) immediately upon reaching the login page. The identify request included a stateHandle property that must have been derived from an earlier request. Passing credentials directly to the identify request without the stateHandle property resulted in an error that the stateHandle was missing.

Sure enough, starting the capture from the login page load revealed additional calls needed to complete the authentication process. As you can see in the screenshot above, the stateHandle value is derived from the introspect request. But introspect had its own input parameter - interactionHandle.

Further up the chain was an interact request which takes URL-encoded form data including a client ID, scope, and redirect URI - all easy enough to copy. However, it also takes code challenge, challenge method, state, and nonce. Initially, I copied these fields into Bruno and fired off a request. Sure enough, I received an interaction_handle that I could pass into the introspect route which then provided the stateHandle value I needed to pass into identify. Progress!

The identify route returns a large JSON payload that includes a successWithInteractionCode element with a nested interaction_code attribute if it succeeds. With this in hand, I was able to query token, the final Okta route on the authetnication journey. This route takes several URL encoded form fields including a code_verifier used server-side to verify the integrity of the connection - presumably that the one who initiated the request with the code challenge also possesses the corresponding code verifier, and the process hasn't been middled along the way.

With a access, refresh, and id tokens in hand, I had what I needed to submit the request to the utility portal's activate route. This failed on the first few tries. It appears in addition to needed a valid payload, certain custom headers were also required.

With authentication complete, I could now focus on the requests for bill downloads. The download request relied on session data stored in the cookie on my machine and an additional Oktauid header value which was retrieved from the identify route earlier. Similar to the activate request, additional headers were also required. But finally, I had a base64 encoded payload of the PDF bill in a JSON object.

A quick test to ensure I could decode it and read the file:

$bill = '{"bill":"..."}' | convertfrom-json
$billExport = [Convert]::FromBase64String($bill.bill)
[IO.File]::WriteAllBytes("test.pdf", $billExport)

Success!

Now, to automate the process. Note that we preserve the session data in a $session variable throughout this process. This ensures by the time we request the bill, the required sessiond data is passed into the request.

$cred = Get-Secret "api_sce"
$oauthEndpoint = "aux123jkl456..."  # Find this in the browser DevTools output in the /interact request
$clientId = "abc123mno456..."  # Find this in the browser DevTools output in the /interact request
$accountnumbers = @(
    "001",
    "002",
    "003"
)

function New-CodeChallenge{
    $bytes = New-Object byte[] 32
    [System.Security.Cryptography.RandomNumberGenerator]::Fill($bytes)

    $codeVerifier = [Convert]::ToBase64String($bytes).
        TrimEnd('=').
        Replace('+','-').
        Replace('/','_')

    $sha = [System.Security.Cryptography.SHA256]::Create()
    $hash = $sha.ComputeHash([Text.Encoding]::ASCII.GetBytes($codeVerifier))

    $codeChallenge = [Convert]::ToBase64String($hash).TrimEnd('=').Replace('+','-').Replace('/','_')

    $bytes = New-Object byte[] 32
    [System.Security.Cryptography.RandomNumberGenerator]::Fill($bytes)
    $nonce = [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+','-').Replace('/','_')

    $bytes = New-Object byte[] 32
    [System.Security.Cryptography.RandomNumberGenerator]::Fill($bytes)
    $state = [Convert]::ToBase64String($bytes).TrimEnd('=').Replace('+','-').Replace('/','_')

    return @{
        verifier = $codeVerifier
        challenge = $codeChallenge
        nonce = $nonce
        state = $state
    }
}

function New-SCEAuth{
    Write-Output "SCE auth: starting OAuth/OIDC interaction flow."
    $challenge = New-CodeChallenge
    $interactUrl = "https://okta.sce.com/oauth2/$($oauthEndpoint)$/v1/interact"
    $interactBody = @{
        "client_id" = $clientId
        "scope" = "openid profile email offline_access okta.myAccount.phone.manage okta.myAccount.phone.read okta.myAccount.email.manage okta.myAccount.email.read"
        "redirect_uri" = "https://www.sce.com/login/callback"
        "code_challenge" = $challenge.challenge
        "code_challenge_method" = "S256"
        "state" = $challenge.state
        "nonce" = $challenge.nonce
    } 
    try {
        $interact = Invoke-RestMethod -Uri $interactUrl -Method Post -Body $interactBody -SessionVariable session -ErrorAction Stop
    }
    catch {
        Write-Output "SCE auth: INTERACT request failed. Error: $($_.Exception.Message)"
        throw
    }


    $introspectUrl = "https://okta.sce.com/idp/idx/introspect"
    $introspectBody = @{
        "interactionHandle" = $interact.interaction_handle
    } | ConvertTo-Json -Compress
    $introspectHeaders = @{
        "Content-Type" = "application/json"
    }
    try {
        $introspect = Invoke-RestMethod -Uri $introspectUrl -Method Post -Body $introspectBody -WebSession $session -Headers $introspectHeaders -ErrorAction Stop
    }
    catch {
        Write-Output "SCE auth: INTROSPECT request failed. Error: $($_.Exception.Message)"
        throw
    }

    $identifyUrl = "https://okta.sce.com/idp/idx/identify"
    $identifyHeaders = @{
        "Content-Type" = "application/json"
    }
    $identifyBody = @{
        "identifier" = $cred.UserName
        "credentials" = @{
            "passcode" = $cred.Password | ConvertFrom-SecureString -AsPlainText
        }
        "stateHandle" = $introspect.stateHandle
    } | ConvertTo-Json -Compress
    Write-Output "SCE auth: identifying as $($cred.UserName)."
    try {
        $identify = Invoke-RestMethod -Uri $identifyUrl -Method Post -Body $identifyBody -WebSession $session -Headers $identifyHeaders -ErrorAction Stop
    }
    catch {
        Write-Output "SCE auth: IDENTIFY request failed (user: $($cred.UserName)). Error: $($_.Exception.Message)"
        throw
    }

    $tokenUrl = "https://okta.sce.com/oauth2/$($oauthEndpoint)/v1/token"
    $tokenHeaders = @{
        "Accept" = "application/json"
    }
    $interactionCode = ($identify.successWithInteractionCode.value | Where-Object {$_.name -eq "interaction_code"}).value
    $tokenBody = @{
        "client_id" = $identify.app.value.id
        "redurect_uri" = "https://www.sce.com/login/callback"
        "grant_type" = "interaction_code"
        "code_verifier" = $challenge.verifier
        "interaction_code" = $interactionCode
    }
    Write-Output "SCE auth: exchanging interaction code for tokens."
    try {
        $token = Invoke-RestMethod -Uri $tokenUrl -Headers $tokenHeaders -Method Post -Body $tokenBody -WebSession $session -ContentType "application/x-www-form-urlencoded" -ErrorAction Stop
    }
    catch {
        Write-Output "SCE auth: TOKEN request failed. Error: $($_.Exception.Message)"
        throw
    }

    $activateUrl = "https://prodms.dms.sce.com/security/v2/activate"
    $activateBody = @{
        accessToken = $token.access_token
        refreshToken = $token.refresh_token
        idToken = $token.id_token
    } | ConvertTo-Json -Compress
    $reqGuid = (New-Guid).guid
    $activateHeaders = @{
        "User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/149.0.0.0 Safari/537.36"
        "Origin" = "https://www.sce.com"
        "X-Is-Secure" = "false"
        "content-type" = "application/json"
        "X-Request-Id" = $reqGuid
        "Referer" = "https://www.sce.com/"
    }
    Write-Output "SCE auth: activating DMS session."
    try {
        $activate = Invoke-RestMethod -Uri $activateUrl -Headers $activateHeaders -Method Post -Body $activateBody -WebSession $session -ErrorAction Stop
    }
    catch {
        Write-Output "SCE auth: ACTIVATE request failed. Error: $($_.Exception.Message)"
        throw
    }
    if($activate){
        Write-Output "SCE auth: session activated (oktaUid: $($activate.oktaUid))."
        return @{
            session = $session
            oktauid = $activate.oktaUid
        }
    }
    else {
        Write-Output "SCE auth: activation request returned no result."
        return $false
    }
}

Write-Output "Starting SCE bill download for $($accountnumbers.Count) account(s)."

try {
    $session = New-SCEAuth
}
catch {
    Write-Output "SCE authentication failed. Exiting. Error: $($_.Exception.Message)"
    exit 1
}

if (-not $session -or -not $session.session -or -not $session.oktauid) {
    Write-Output "SCE authentication did not return a valid session. Exiting."
    exit 1
}
Write-Output "Authenticated to SCE (oktaUid: $($session.oktauid))."

$succeeded = 0
foreach($account in $accountnumbers){

    Write-Output "Downloading bill for account $account ..."
    try {
        $billUrl = "https://prodms.dms.sce.com/billing/v1/pdf/$($account)"
        $reqGuid = (New-Guid).guid
        $billHeaders = @{
            "Oktauid" = $session.oktauid
            "User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/149.0.0.0 Safari/537.36"
            "X-is-Secure" = "false"
            "X-Request-Id" = $reqGuid
            "Referer" = "https://www.sce.com/"
            "Origing" = "https://www.sce.com"
            "Accept" = "application/json, text/plain, */*"
            "Content-Type" = "application/json"
        }
        $bill = Invoke-RestMethod -Uri $billUrl -Headers $billHeaders -WebSession $session.session -ErrorAction Stop
        $billExport = [Convert]::FromBase64String($bill.bill)
        [IO.File]::WriteAllBytes("$($account).pdf", $billExport)
        Write-Output "Saved bill for account $account to $($account).pdf ($($billExport.Length) bytes)."
        $succeeded++
    }
    catch {
        Write-Output "Failed to download bill for account $account. Error: $($_.Exception.Message)"
        continue
    }
}

Write-Output "SCE bill download complete. $succeeded of $($accountnumbers.Count) bill(s) downloaded successfully."