By
@azure
ai captcha solver using pyautogui and yolo! cant reliably host this all you need is a google collab and local computer but cordinates are for my mointor, but because of google collab cannot host a demo as links change everytime and i cant keep collab side up limited compute time
i made a captcha solver using yolo and pyautogui. one of the biggest challenges was compute, i don’t own a GPU so trying to run the model locally wasn’t possible but google collab was giving me issues even when tunnels worked sometimes it wouldn’t bind to port. I’m proud of how it turned out in the end not the most accurate project but a fun make. you’ll need to clone the project and put notebook into google collab or modify it for local use when the notebook works just copy the 1st link you see and paste it into the URL in the autocaptcha.py
1h 8m 11s logged
devlog 2
fixxing ocr and double click cordinates. I realized that it wasnt a traing issue most of the time rather the ocr just sucks so i cropped the header to help. and i also had client side cordinates for the header so ocr was bugging i fixxed it by adjusting for its pov. same with click cordinates. also for some reason it clicks the same square twice to i decied to filter out over lapping classes to fix it using iou. it has improved the accuracy but still not as accurate as id like
1h 8m 49s logged
devlog 1
adding on fixxing the tunnel not connecting to port 8000/ 502 gateway error i needed a time .sleep after debugging with curl, also. ocr struggles with all the text so i cropped it for purely ocr, with a test script to find each corner and the width/height of it. need to fine tune model for more items as of now it doesnt know what to click and is only good at items in dataset and the full item not pieces of it.
