most of the parsing issues have been fixed! the final list is 1321 items, which is the same amount as previous scrapes, so i don’t think there are any issues there.
a couple things still worry me a bit (but they’re not things that hinder progress elsewhere):
- the build command lingers after it says it’s finished building. i suspect it’s because some promise ends up never resolving, but i’ll have to look into that further
- the specifications field is messed up. i should also add a string sanitizer later on to the helpers file, but in the migration to cheerio something with the specification parsing logic got messed up.
- a cached build still takes two minutes. ideally i would like it to be more around the 30 second range, but i have to figure out if there are any bigger things worth caching.
there’s just a type error i have to deal with when it comes to error handling, and everything should be good! i’ll move on to more site prototyping since i have a dummy database to test things against.
Comments 0
No comments yet. Be the first!
Sign in to join the conversation.