A weekend project transformed into a 4-day project: Fortnite OCR Detector. OCR stands for Optical character recognition.

A fellow streamer needed a way to show his number of victories and kills for the day above his webcam in order to add something unique for his audience.

For this project, images are provided as input by taking screenshots of the entire monitor where the game resides (otherwise it's seen as a hack and you'll get banned), cutting the image and only getting the sections of the screen where the kills reside, as well as some other sections to detect when a victory was made and update the counters accordingly.

I wanted to keep things separate, thus the OpenCV library was used in a C++ shell app 'Detector' that was in charge of detecting the above said. Most of the time was spent here, changing and adapting images and numbers and algorithms to give the best accuracy for the numbers.

I then had a python process that was bringing many things together:

1. instantiate a child process 'Detector', read data and update the counters.
2. create a simple local HTTP web server that delivers an 'index.html' file at 127.0.0.1:5511, as well as the custom font at 127.0.0.1:5511/font.{ext}.
3. Deliver the index.html file would also create a client to a local WebSocket server, and reconnect in case the connection is dropped. It has to be backwards compatible to IE6 so the streaming application OBS is capable of loading the local page, replace the colour 'white' with 'transparent', and display it on top of the game.
4. create a simple WebSocket server that updates the clients (the browsers in this case) with the most up to date counters using Facebook's tornado
5. create a window using the wxpython library, a cross-platform freeware GUI application maker.

The python script was converted to windows executable using pyinstaller.

The application running the OpenCV detection algorithms and the number-crunching takes 0% GPU and <2% CPU on an Intel Core i7 4790 3.6GHz box.