You are browsing as a guest. Sign up (or log in) to start making projects!

OvODoc

  • 4 Devlogs
  • 14 Total hours

A web Scrapper for anime/larp/cosplay convents in poland

Open comments for this post

3h 49m 3s logged

I’ve added an option to filter by event locations :pochita-plush: :pochita: (I love pochita ❤️ )

so yeah not much to say ;_; you can filter by location but it took me a fair bit of time still I’m proud af bc I went from not knowing anything about nextJS and React to actually doing and making something with it :yipee:

technical stuff:
So the first thing I did was to modify the python api to store unique locations since I know more about python and I wasnt sure how to do it on frontend so I figured I might as well just use python for it


and as for actually displaying it when a user clicks a year, it updates the ?year= URL parameter. The renderTable.tsx component detects this URL change and fetches both the table events and the unique cities from the API. These cities are then saved to the cityData state and passed down as a prop to the DataTable component. Inside data-table.tsx, this cityData is fed into the Combobox to display the list of available cities.

0
0
5
Open comments for this post

1h 45m 15s logged

I actually DONT HATE doing front end

yeah so after yesterday I’m starting to actually know what I am doing :yayy:

What I’ve added

So today I did two main things:

  • search bar
  • overall ui changes to make it look a little bit more tolerable to look at (imho it looks wayyyyyy better and I actually like it and for my first attempt at a proper front end I’d say this looks good)

So I continued to use retro ui but this time I understood (kind of) how to use it and the process was wayy more enjoyable plus I’ve spend some time getting to know tailwind which made the process even better

As for the searchbar… I have no idea how it works but it works I followed the random tutorial I’ve found

I’ve spend suprising amount of time on tweaking and trying a lot of random thing like trying to put buttons in between the input and the table or playing around with diffrent sizes and colors

What next

tomorrow I will be adding the ability to filter the events by cities and maybe by diffrent things :idk: idk yet

0
0
4
Open comments for this post

1h 4m 25s logged

I hate doing the front end

I have no idea what I am doing :crying:

yea so I tried to make the table look like the ones in retro ui without the tutorial (big mistake) but after trying a lot of things I ended up with something like this ???? imho it doesnt look bad at all


I will try to style the buttons somehow tmrw wish me luck

0
0
2
Open comments for this post

7h 31m 15s logged

So I wanted to finish this project at the end of the last week to get some stickers but as you can tell I failed horribly to meet that deadline. So I wanted to make a simple web scraper that would gather info about anime conventions and a simple frontend for it (why did I choose this well my friend REALLY likes attending those events and he was constantly talking about it nad it got kinda stuck in my head and since the website that he uses to find those events is quite old I decided to build something new) but as it turned out the front end wasnt easy at all bc I’ve never used nextJs before.

WHY DID I DECIDE TO USE NEXT EVEN THO I HAVE NO IDEA WHY TO USE IT

So why did I decide to use it then wellllll I was talking to my friend on discord and I asked him how do I make a simple front end quickly bc the deadline was approaching and he told me “oh yeah just use shadcn and nextJs”… .

THE BACKEND: Python, Selenium, and Flask

So I starded building the web scraper which im actually quite okay at since I’ve done that before. I’ve used Selenium for navigating the website and BeautifulSoup for scraping the data which I was saving to json and using Flask for making the api and that was it the only a bit tricky thing was that the buttons on the website were acting as filters for some reason ??? but once I figured it out it was all good.

FRONTEND…(it was bad)

And for the front end uhmmmm yeah it is VERY bad and it took me wayyy longer then it should I never could have imagined that something as simple as building a table with some data and changing it based on a year could be this hard… At first I decided to play around with nextjs and use states which went quite good I kind of understood them (at a very basic level) then the struggle came rendering the buttons (they dont even look like buttons lmao) dynamicly from the api. Since I’ve never used TypeScript, Next.js, React and Tailwind I had to learn everything from scratch like props and everything else but once I figured props out it the buttons were easy to make. Then came time for the table which went really smooth since it was just a shadcn table. But I wasnt prepared at all for the next part which was changing the contents of the table after clicking the year button (more like text bc it doesnt look like a button at all) So after many different approaches I came up with this:

  1. The user clicks a year.
    2)The app creates a URLSearchParams object from the current query string after that it adds or updates the year parameter and builds a new URL amd it replaces the current URL using window.history.pushState.
    3)The getData function fetches the new data from the API.
    4)My renderTable.tsx component catches the new data and updates the table.

WHAT WILL I DO NEXT

And yeah thats it… it was way harder then I thought it would be but I learned something about nextJs I guess and yeah I will be probably continue to work on it and use it as a project to learn nextJs on

0
0
5

Delete project?

Are you sure you want to permanently delete this project? This action cannot be undone.

All devlogs, followers, and associated data will be removed.

Followers

Loading…