The Twitter Bot That Help You With French Consulate Appointment!Thu, 28 Apr 2022
#serverless #sst #twitter #api #scraping
This article have been written in early April 2022. Since the release of the bot it seems that the underlaying API have changed a bit. I am not planning to document the modification of the bot that have been done after this initial version, but it will give a pretty good picture of the methodology used.
I am currently living in Montréal, QC but I am still a French citizen. So like thousands of French citizens living abroad, I rely on the French Consulate for a few administrative procedures, including passeport renewal. And in the last years, it has been more and more complex to book an appointment in order to renew or ask for a passeport:
No appointment available on French Montreal Consulate as of today...
So my plan was to automate the retrieval of available appointements in order to be able to book a slot (for my wife), and send it to a tool where a notification system is available, and Twitter seems a good candidate.
This article will guide you thought this process: first you will understand how the application is working (this one), then you will build a serverless application using Serverless Stack that will extract the needed information and send it to Twitter.
A few word on the used tools
The appointment application
The French Montreal Consulate is relying on a third-party application to manage the appointment system:
There is a big TROOV logo on the page, and a few indicators in the source code linking to TROOV company. TROOV is providing an online appointment booking solution named TROOV TON CRÉNEAU:
I am using a lot this toolkit to build solutions that will run on an AWS account for a very low amount of $. It’s a very elegant all-in-one solution to manage serverless applications, you can read it more here:
- SST: The Most Underrated Serverless Framework You Need to Discover
- SST: The Most Underrated Serverless Framework You Need to Discover (part 2)
- Why and How Migrate From Firebase to Serverless Stack?
Twitter is providing an API to build bots easily. I have been inspired by the Vaccélérateur project here in Québec: a very simple Twitter bot that share the next slots for a COVID-19 vaccine shot in your area:
Understand the application
The first step here is to understand if it’s possible to grab programmatically the appointment list for the Consulate. Here my best tool is the Developer Tools menu that you can open in any decent browser nowadays (generally with
F12). Here is the one I have on Chrome:
Some important points:
- Let’s focus on the Network tab, it give you the various REST API calls done by the frontend application to the backend.
- Let’s filter on
Fetch/XHRto have only REST calls, and don’t see images, css, and other files that are not useful now
- If I click on
exclude-daysthen it give you on the left all the information regarding this specific API call. You have the headers, the payload sent to the backend, and the preview/response sent by the backend to the frontend. Here for exemple I am viewing a list of dates, sent by an endpoint called
exclude-days. 🧐 So we can safely imagine that this is a list of day with no availability at all.