PHP Application to crawl, retrieve, archive and compile several internet banking raw data from different Indonesian local banks. This application was scheduled to routinely get internet banking raw data using cURL. Those raw data would then be parsed using PHP Simple HTML DOM Parser to convert raw HTML data (mostly in the form of traditional HTML table) into a structured data. Finally, these structured data would be saved and archived to the database (using MySQL database) for later use.
This application is able to automatically categorize each transaction by examining the sender name and the pattern of the transaction description. This automatic categorization is very useful for me to analyze my own incomes and expenses.
The main challenge with building this kind of application is of course tackling any kind of restriction from the banks. Many banks implement many kinds of restriction on their internet banking system to prevent robot to grabs the data. They might implement sophisticated Captcha, session based URL, HTML obfuscation etc. Also each bank has their own presentation of raw html data. This require me to build a special parser for each one of them.