[Due for payment 2025-04-02] [$500] Set up WebRTC connection for real time conversation

[thienlnam]

Internal issue: https://github.com/Expensify/Expensify/issues/482733

We're exploring the potential an AI SDR, and want to set up WebRTC on web to test this out

Requirements:

New API endpoints
GetEmphemeralToken
SendRecapInAdminsRoom

• reportID
• recap

1. Add support for WebRTC on web
2. Create a method called connectToOpenAIRealtime which will call GetEmphemeralToken and connect to the OpenAI real time endpoint

const response = await fetch('https://api.openai.com/v1/realtime', {
                method: 'POST',
                headers: {
                    Authorization: `Bearer ${ephemeralToken}`,
                    'Content-Type': 'application/sdp',
                },
                body: offer.sdp,
            });

2. Add a method to stopConnection which will close the webRTC connection
3. Add handleFunctionCall which will handle the model function calls. This should currently support EndCall, and SendRecapInAdminsRoom
4. Add a new beta newDotTalkToAISales
5. Add a ☎ Talk with sales button that will call connectToOpenAIRealtime. This button should be on the left of Schedule demo, and only show up if you are behind the beta
6. During the call the button should switch to Hang up

Note: Here's some code to get you on the right track https://github.com/Expensify/App/pull/56528/files, just needs to be cleaned up and correct types added (ignore the side pane and screen recording stuff)

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~021902790241903311523
• Upwork Job ID: 1902790241903311523
• Last Price Increase: 2025-03-22
• Automatic offers:
• DylanDylann | Contributor | 106604743

[thienlnam]

Note: the endpoints are currently in review:

GetEmphemeralToken
SendRecapInAdminsRoom

So I can pull the App PR to test when it's ready

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~021902790241903311523

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @parasharrajat (External)

[melvin-bot]

📣 @DylanDylann 🎉 An offer has been automatically sent to your Upwork account for the Reviewer role 🎉 Thanks for contributing to the Expensify app!

Offer link
Upwork job

[melvin-bot]

Upwork job price has been updated to $500

[rushatgabhane]

hey, i really appreciate the price bump <3

[melvin-bot]

⚠️ Looks like this issue was linked to a Deploy Blocker here

If you are the assigned CME please investigate whether the linked PR caused a regression and leave a comment with the results.

If a regression has occurred and you are the assigned CM follow the instructions here.

If this regression could have been avoided please consider also proposing a recommendation to the PR checklist so that we can avoid it in the future.

[melvin-bot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[melvin-bot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 9.1.18-4 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• [WebRTC] Make a call to AI SDR (Sales & development representative) #58867

If no regressions arise, payment will be issued on 2025-04-02. 🎊

For reference, here are some details about the assignees on this issue:

• @rushatgabhane requires payment through NewDot Manual Requests
• @DylanDylann requires payment automatic offer (Contributor)

[melvin-bot]

Issue is ready for payment but no BZ is assigned. @johncschuster you are the lucky winner! Please verify the payment summary looks correct and complete the checklist. Thanks!

[johncschuster]

Payment Summary

Contributor: @rushatgabhane owed $500 via NewDot
Contributor+: @DylanDylann paid $500 via Upwork

[johncschuster]

@DylanDylann please accept the job invite so I can issue payment. Thank you!