Devlog 16

I just boosted the security of the tool quite a bit, now with rate limiting, aggressive schema validation, and payload size limits. The issue before was that you could spam stuff no problem, but now there is a ton of resistance, and very strict limits. I still have more plans to make it even more secure as well. I also setup meta tags so now supposedly when you send a link depending on the platform, it will show this image and a description as well.

An example of some of the new aggressive schema validation is:

class QuestionChanges(StrictModel):
    title: Str | None = None
    type: (
        Literal["ln", "sn", "cb", "a", "img", "n", "mc", "sc", "r", "st"]
        | None
    ) = None
    opt: dict[Annotated[str, StringConstraints(pattern=r"^[0-9]$")], Annotated[str, Field(max_length=50)]] | None = Field(default=None, max_length=10)
    minmax: Annotated[list[Annotated[int, Field(ge=-99999, le=99999)]], Field(min_length=2, max_length=2)] | None = None
    stars: Annotated[int, Field(ge=0, le=10)] | None = None

I setup two different rate limiting systems, these include ones for the websocket, where each individual message type can get its own rate limit (some things should be easily called 500 times in a minute, while others should NOT be called more than 10)

@ws_limit(maxCalls=500, window=60)
async def handleUpdateTeamQuestion(

@ws_limit(maxCalls=8, window=60)
async def handleCompCodeChange(

Window being time in seconds, maxCalls being the max calls in that window.

This still needs more testing to make sure edge cases that are actually possible to do using the UI are not blocked by this security, but most regular use seems to not be affected.

Devlog 15

I have now finished the following tasks:

• Inviting members
• Uninviting members
• Kicking members out of group
• Most of the UI to do with the member system (only joining a group left)

I have also added these improvements to the backend related things:

• The front end no longer assumes that a user is in a group and instead calls a server endpoint to check first on sign in
• The front end now connects on loading of the dashboard page instead of loading on the sign in page, which is to fix a current issue where if the user is already signed in but on the landing page, it makes them re-sign in, but soon it will just be a button saying “go to dash”
• Added UI for loading hydration for dash
• Made it so the add match/add team buttons were only visible to admins

I now pretty much don’t have too much to do, instead it will simply be finishing touches and a little bit of work with the joining system:

• Make the UI for creating a group
• Make the UI for joining a group actually work
• Add rate limiting
• Sync settings with database
• Publish

Devlog 14

I have now fully finished the summary team page UI as well. Now there are charts that show the points of each individual match, and then also each individual method of scoring.

I also added two new screens for when a team is in first or last, if a team is in first, then there are no teams above to accept/reject, so it shows a little bit of text, if a team is in last, there is no team below to pick, so they have a separate message.

I also took the time to add the flags bit beside a team’s name in the summary page too so it can match prescout.

With all of the above, I also completed the mobile UI equivalent.

Devlog 13

ALL. SERVER. SYNC. IS. DONE.

I have finished every single piece of work needed to sync the server with the client at all times. I am now getting close to the finish line for this project.

I added info toasts for some new things, rather than just showing the success toast for an action a user may not have actually done themselves for things like adding teams and matches.

There are now very few tasks left really, they are:

• Fully localize everything
• Add French
• Add mobile UI to summary
• Add custom competition handling for summary
• Add a favicon
• Fix dark mode glitches on sign in/sign up pages

Devlog 12

This devlog is now much more about the server side of things as that has been my focus right now.

The way this all works is I have a realtime python server connect to my database and all clients. Clients are put into a realtime channel with all their teammates. When the first user of a team joins, the python server pulls all that team’s scouting data from the database and keeps it in memory. This is to speed up request response times. When a second, third, fourth, so on, user of a team joins, the server instead just gives them the data it has in memory instead of the database. To keep this all in sync, the server updates its version of that group’s data as soon as it is successfully saved to the database.

It takes time to set this all up, and the parts of synchronization that I have completed so far are:

• Competition code changes
• Setting competitions to custom
• Adding an individual team
• Adding teams from API at the start
• Adding an individual match
• Adding matches from API at the start

I have done one thing for the UI though, and that is finally adding a 404 page. I designed this 404 with the magnifying glass in the middle as the 0.

Devlog 11

I usually space out my devlogs more, but I have now managed to get the base connection. This went by rather flawlessly due to now having the ability to use a python server for websockets, rather than the old system of having to use SQL requests with a million different RPC functions (sometimes causing queries to take up to 500ms on average) now taking only 100ms on average.

This uses a simple system to take the JWT token from the user, validate it from the server to find the user id, then checks the database to see what group the user is in, then checks the group to make sure that the user is actually in that group, and if all those conditions are met, the user will connect to the websocket.

If the user is the first one in the websocket, it right now just prints it, but it is designed to soon be used for a new feature. Soon, when a user joins a websocket and is the first one there, the server will load all that group’s data from the database, the reason for this is to have lightning fast connection and also make it possible to synchronize all users’ data in real time. (Whenever a user makes a change, it would update the server’s copy, and also push that change to the database)

Devlog 10

I have now finished the skeleton UI for the summary tab.

I originally ran into some issues because I had tried to make one table with one map and just change the data for it, but that went horribly wrong because it made the code so janky and I had to write a ton of spaghetti code for it. In the end, I decided to switch to this system instead, where there are 3 completely separate tables for each tab, that get rendered in conditionally.

{currentTab == 0 ? (
    <Tab1
        sorted={sorted}
        sortBy={sortBy}
        setSortBy={setSortBy}
        setSortDown={setSortDown}
        sortDown={sortDown}
        selected={selected}
    />
) : currentTab == 1 ? (
    <Tab2 teamsBelow={teamsBelow} />
) : (
    <Tab3 teamsAbove={teamsAbove} />
)}

This way allowed me to have more freedom and more organization with the code.

I also originally ran into an issue with my rendering system for picks. When a team would set their pick order, originally to preserve the order they had in mind, the site would check all teams in the picks array, and if they were below, they would render them in. This caused an issue where if a team changed ranks through a competition, the teams below would change, and some might not render, which caused a problem in that the indexes for the reordering were off, and it would reorder the ones at the top that were not visible to the user. In the end I fixed this by simply filtering out and deleting the ones that were no longer a valid pick.

updateSummary({
    picks: [
        ...summary.picks.filter((pick) => below.includes(pick)),
        ...below.filter((team) => !summary.picks.includes(team)),
    ],
});

My goals now are to get a start on the backend part so that the main structure of the site can be complete.

The tasks that would be left after that are:

• Polishing the summary page UI
• Making the mobile UI for the summary page
• Adding the new setting to set user’s team number by default so they don’t have to constantly set it from the tab
• Landing page improvements

Devlog 9

I have now done a lot of work in relation to the summary page, the hardest part is now pretty much done (the all teams page).

The parts left now are the top picks page, and the accept/reject page.

I have also done some work that isn’t specifically for the summary page too. Now, setting a competition will make it so the FTCScout api will actually pull the teams and matches in the competition.

The only steps left before synchronization now are those previously mentioned summary pages, and the mobile UI for all summary pages.

Devlog 8

This is now where things are really starting to pick up. With the entire match scouting page done, there is now only summary, and then it is time to work on the synchronization.

I was thinking about leaving the score page empty, but instead I decided to just put what would be there for FTC Decode, I will later switch it to Biobuzz when the game is known.

Devlog 7

I have now finished the team prescout page for both phones and desktop, this is where scouts can fill in those questions from earlier. I didn’t really run into much issues, but I implemented a special system to make sure that this fill in works with my current question structure.

The original issue

The original issue with this system is that if a question is edited, it keeps its question id, and if a user edits the question type, that means that what was once a multiple choice, can now be a note, which would completely mess it up if a user opens a team to check on the data. The solution? If a user enters a team page with an invalid question, the question will be reset to its default, so a string will be reset to “”, a number input will be reset to 0, etc.

Now this means the prescout page is pretty much completely done, and I now have to start on the match scouting page.

Devlog 6

I have now finished the phone UI for the prescouting page, meaning the last step of this page is officially complete. The next tasks are the individual team pages and the start of the match scouting page.

Devlog 5

I have now finished the following tasks:

• Ability to edit question extra fields (e.g. min/max of a slider question)

• Ability to delete a section

• Ability to add a section

• Ability to edit a section

• Ability to add a question

The tasks I have left related to the question management system are:

• Translation keys

• Create the phone UI

I also hit a small hiccup along the way of adding these features where I discovered due to my localstorage structure, whenever a competition is switched, the questions are all deleted (now fixed by not just using a blank localstorage.clear())

New Code

/**
 * Creates the base skeleton structure for localstorage
 *
 * @param {boolean} force - Force overwrite of existing localstorage or not
 */
export function createSkeleton(force: boolean) {
    const existing = localStorage.getItem("data");

    if (existing && !force) {
        return;
    }

    let setPrescout = {
        structure: {},
        sections: {
            "0": {
                title: "Section 1",
                headersize: 1,
                questions: [],
                index: 0,
            },
        },
    };

    if (existing) {
        const parsed = JSON.parse(existing);

        setPrescout.structure = parsed.prescout.structure;
        setPrescout.sections = parsed.prescout.sections;
    }

    const skeleton: LocalStorageData = {
        compkey: "",
        custom: false,
        prescout: {
            structure: setPrescout.structure,
            sections: setPrescout.sections,
            teams: {},
        },
        match: {},
    };

    localStorage.setItem("data", JSON.stringify(skeleton));
    resetAllStates();
}

Devlog 4

I have spent almost 8 hours working on one feature and one feature alone. That feature is this drag and drop organizer for questions. This lets the user drag around the order of questions to rearrange them, and also drag around the order of sections.

The reason this took so long was because the original library I was using (dnd-kit) for drag and drop made it a complete nightmare to try and do anything, it did not have a way to tell what items were where, as it would literally just change the HTML rather than give any data. Once I switched to hello pangea dnd, it all started fitting together. It actually gave me a way to see what changes occurred, save them, and then that updates the UI.

Now my next tasks are:

• Ability to edit question extra fields (e.g. min/max of a slider question)
• Ability to delete a section
• Ability to add a section
• Ability to edit a section
• Ability to add a question
• Translation keys
• Create the phone UI

Devlog 3

I have learned my mistake, so now I will be aiming for a devlog every 3-4 hours instead.

I have now made it so that the progress bar for teams scouted actually shows the real progress so far of scouting, and the lights actually react to that team’s scouting progress. (I have not made the system to add questions yet, so technically all teams are fully scouted (0/0 questions answered)).

I have also started work on the question organization menu. It will be a drag and drop menu to let users arrange and edit questions how they want, with a button at the bottom to edit questions.

The images attached below show the progress bar and drag and drop menu (ignore the dummy data put for now, I will replace that with actual data later), and the dropdown menu, which will be for selecting question type soon. I have also attached a very rough sketch of what the draggable question and section divs themselves will look like. (the blue and red is just to indicate different colors, it is not the final color)

Devlog 2

So far I have now completed the main job of the competition tab, which is
to handle adding custom matches and teams. One of the major issues I came along
was that the UI was ugly, the modals were bland, and I wanted to add more depth, so I added that highlight in the middle which lifts the interactable content up. I had issues with the height of the modal being either too tall or too short depending on the device screen. I solved this one by making a simple new function that helps decide if the modal needs to be bumped up in height or not

/**
 * Returns whether the current screen height is "akward", <= 700px for phones/tablets, and <= 899px for desktop
 *
 * @returns true | false
 */
export function useIsAkwardHeight(): true | false {
    if (useScreenType() != "desktop") {
        const isAkwardHeight = useMediaQuery({ query: "(max-height: 700px)" });
        return isAkwardHeight;
    } else {
        const isAkwardHeight = useMediaQuery({ query: "(max-height: 899px)" });
        return isAkwardHeight;
    }
}

An example of it being used:

<div
    className="desktop-warningpopup"
    id="avoidwarningpopupheight"
    style={
        specifyCustomCountry
            ? { height: useIsAkwardHeight() ? "55vh" : "40vh" }
            : { height: useIsAkwardHeight() ? "40vh" : "25vh" }
    }
>

One big thing I have done in the past bit of time (I did not know that anything after 10 hours is voided, meaning I just wasted like 5 hours 😭) is the organization of a lot of the code.

I also organized a lot of the CSS into barrel files so that the CSS files are not thousands of lines long anymore.

I have also started work on the prescouting tab (finally) and with that I have got the UI for the progress bar of teams scouted, and also the table of teams.

One big thing I have now put in this project is persisting settings, and one big settings function.

export const useSettings = create(
    persist<{
        isLightMode: boolean;
        flipTheme: () => void;

        isCustomCountry: boolean;
        flipCustomCountry: () => void;
    }>(
        (set) => ({
            isLightMode: true,

            flipTheme: () => {
                set((state) => ({
                    isLightMode: !state.isLightMode,
                }));
            },

            isCustomCountry: false,

            flipCustomCountry: () => {
                set((state) => ({
                    isCustomCountry: !state.isCustomCountry,
                }));
            },
        }),
        { name: "settings" },
    ),
);

Devlog 1

This is my first devlog here, so I will explain a little bit about what this project is. Scoutmaster is a First Tech Challenge scouting tool designed to make scouting at competitions easier for teams. It has prescouting, which is basically scouting teams’ abilities before matches start to see what features they may have that might not be seen in matches, and then match scouting which lets scouts figure out exaclty how many points on average each team scores on their own, to make alliance selection at the end much easier.

The original Scoutmaster did that, but in a very primitive way, the UI was not that good, and there were many bugs. This prompted me to rebuild it from the ground up, and fix those bugs along the way with what I have learned since then.

So far, I have put in about 12 hours into this project. Within this time, I have managed to get a start on:

• Landing Page for both Desktop and Phone
• Dashboard for Desktop and Phone
• Initial API structure

{
    "compkey": "CAONBOQ",
    "custom": false,
    "prescout": {
        "structure": {
            "numOfQuestions": 6,
            // the type of questions
            "questionorder": ["lt", "st", "cb", "r", "auto", "picture"],
        },
        "teams": {
            "1": {
                // for images, the image name will be Scouting team number - picof Scouted team number, so XXXXX-picof-XXXXX
                "data": [
                    "lorem ipsum",
                    "situm dilor",
                    true,
                    213,
                    "IMAGINEANSVGPATHRIGHTHERE",
                    "16423-picof1.png",
                ],
                "matchesIn": [1, 5, 67],
            },
            "2": {},
            "3": {},
        },
    },
    "matchscout": {
        "Q1": {
            "highlighted": [true, true, false, true],
            "teams": [1234, 1222, 1223, 1313],
            "red1": [45, 45, 45, 45],
            "red2": [55, 55, 55, 55],
            "blue1": [555, 555, 555, 555],
            "blue2": [1, 1, 1, 1],
        },
        "Q2": {
            "highlighted": [true, true, false, true],
            "teams": [1234, 1222, 1223, 1313],
            "red1": [45, 45, 45, 45],
            "red2": [55, 55, 55, 55],
            "blue1": [555, 555, 555, 555],
            "blue2": [1, 1, 1, 1],
        },
    },
}

This is the initial API structure of what this tool will be using. This is designed to allow for both official competitions and custom ones, a feature not previously included in the original Scoutmaster. The plan now is to continue making the dashboard, and then afterwards, once the full dashboard works with localstorage, begin working on the backend to actually make it synchronize across devices.