First 6DoUD Devlog!

Introduction

Hi there, I’m Toby, and this is the first devlog for my new project titled “6 Degrees of Urban Dictionary” (6DoUD for short).

The title is technically WIP but it’s pretty much here to stay.

This project is based off of (in concept, not code) Jacob Wenger’s 6 Degrees of Wikipedia. If you are unfamiliar with the concept, you choose a starting Wikipedia article and a target article, and the website finds the shortest path between them by following only hyperlinks between pages.

It’s a fairly simple concept but super cool in practice and super fun to mess around with. This got me thinking, what if there was something like 6DoW, but for another website?

The Idea

That ‘other website’ turned out to be Urban Dictionary.

If you’ve never heard of it, Urban Dictionary is a website where users can upload their own definitions for words and phrases. One interesting feature of Urban Dictionary is that, similar to Wikipedia articles, the definitions have hyperlinks that link to other definitions.

This essentially creates a massive interlocking web of definitions similar to the Wikipedia articles.

This eventually turned into the basis of 6 Degrees of Urban Dictionary. Find the shortest path between two definitions by only using the hyperlinks on the definitions in between.

Progress so far

Despite being early in development, the core functionality is already up and running.

At its very core 6DoUD treats Urban Dictionary as a graph. Each term is represented as a node, with the hyperlinks on these definitions connecting these nodes. This allows the website to essentially be modeled as a large network of interconnecting terms. (Of course this graph isn’t actually visualized, my computer would die..)

To build this network I made a crawler which uses the Urban Dictionary API to retrieve definitions and examples for a given term. Using regular expressions, the crawler extracts all linked terms and stores the relationships inside of a SQLite database.

As the crawler discovers new terms, it continues exploring them recursively, gradually building a larger graph of connected entries. To improve performance the crawler runs across multiple threads, allowing many terms to be processed simultaneously.

Once enough data has been collected, the second part of the project takes over: finding the shortest path between two terms. This is done using Breadth-First Search (BFS), a graph traversal algorithm that guarantees the shortest route is found.

At this stage, the core concept is fully functional. Given two terms, the system can search through the collected graph and determine the shortest sequence of links connecting them.

What Next?

While the backend functionality is largely complete, there is still a significant amount of work before the project is ready for end users.

User Interface

Currently, the project is designed primarily for testing and development. The next step is creating an interface that makes the system easy and intuitive to use.

Frontend Development

I plan to build a web frontend inspired by 6 Degrees of Wikipedia. Users will be able to enter a starting term and a destination term without needing to interact directly with the underlying Python code.

Graph Visualization

One feature I would like to implement is a visual graph display. Rather than presenting users with a simple list of terms, the application will display the discovered
route as an interactive network of connected nodes.

API Development

To connect the frontend with the backend, I will also develop an API which exposes the pathfinding functionality. This will allow the frontend and backend to remain separate whilst still communicating efficiently.

Closing

Thanks for taking the time to read through my devlog, attached is a fairly relevant xkcd comic.