Workflow Closure & Collapse orchestrator

This feature ensures the git commit history remains clean from wips, and the closure when the development of a requirement is finished remains frictionless by automating git operations such as path finding, worktree and branch deletion. Leaving the workspace clean and ready to restart the workflow again for a new update or extra feature.

How this engine works

Once the builder has complete the implementation, perform a final commit with all changes implemented and staged When the builder runs the command collapse within the worktree to finish the workflow, Then the engine should:

1. Merge the final commit to the main branch
2. The implementation of the requirement wips are preserved in the final commit squash.
3. The history has only one commit with the wip changes
4. The merge is complete successfully
5. Delete the worktree & branch once the commit is present in the main branch to ensure no data is lost and the req name is availability for future implementations
6. Unlink the symlink connection of the markdown editor for the specific docs folder and link to the main docs tree

The engine in action

Git Wortree Set Up & Spawn Orchestrator

New functionalities

• The spawn command is now connected to the orchestrator in the package.json so that it can later be publish in npm and make it fully global accessible in the developer system by using the -g flag
• Config changes memory: Every time the system changes the system-config.json the changes are automatically staged and commit so that they are present in the new worktree

Testing

• Created two integration automated test using jest that stress test the spawn engine under different scenarios and flags, implement a cool ghost typist so that no questions arise why testing, since those are handle by the ghost typist and test itself.
• Performed an e2e test in an isolated enviroment within my computer, finding some bugs and fixing them properly. (The video shows the e2e test in action)

Fixes

• The system faced a config path mismatch when the command was trigger when the worktree was outside of a prefix folder and then moved to it, making the config file ENOENT.

e2e test

e2e spawn engine in action

1. Create a folder with the prefix if not previously created and move the main worktree dir and other worktrees present there
2. Handle the config-system.json presence if not handle its creation
3. Create a new worktree with the artifact in lowercase as name and branch to work within the folder with the project identifier prefix
4. Connect the docs of the new worktree to the folder in the markdown editor
5. Open a VS code instance

Git Wortree Set Up & Spawn Orchestrator

I develop the initial part of the system requirement of a workflow orchestrator that interacts with git and the system files to create a frictionless, fast and intuitive experience when bringing ideas to live.

Creating and isolated environment that ensures the change log remains pristine and connected to a specific Requirement artifact and its context and market value.

Command and flags

The command that triggers the first part of the orchestrator is spawn with a string argument that will be the name of the requirement to implement e.g: req023. It can be called with different flags to modify the behavior of the orchestrator and achieve different functionalities:

• --symlink or -s
• --code or -c

Flag Combinations and what it achieves

Scenario A: Git Tree Workflow & Symlink Connection To The Markdown Editor Update

Only trigger the command with the -s flag:

1. Creates a folder with the prefix if not previously created and move the main worktree dir and other worktrees present there
2. Create a new worktree with the artifact in lowercase as name and branch to work within the folder with the project identifier prefix
3. Connect the docs of the new worktree to the folder in the markdown editor

Scenario B: Git Tree Workflow Intialization enhance functionality for VSCode instance & Symlink Connection To The Markdown Editor Update

Trigger the command with the -s & -c flag:

1. Create a folder with the prefix if not previously created and move the main worktree dir and other worktrees present there
2. Create a new worktree with the artifact in lowercase as name and branch to work within the folder with the project identifier prefix
3. Connect the docs of the new worktree to the folder in the markdown editor
4. Open a VS code instance

Scenario C: Git Tree Workflow Initialization enhance functionality for VSCode instance

Only trigger the command with the -c flag:

1. Create a folder with the prefix if not previously created and move the main worktree dir and other worktrees present there
2. Create a new worktree with the artifact in lowercase as name and branch to work within the folder with the project identifier prefix
3. Open a VS code instance

Scenario D: Default only worktree creation folder set up

Trigger the command without flags:

1. Create a folder with the prefix if not previously created and move the main worktree dir and other worktrees present there
2. Create a new worktree with the artifact in lowercase as name and branch to work within the folder with the project identifier prefix

Challenges

To handle the underlying mechanism of git such as the way it manages the links from the main worktree to its linked ones, how to move them without breaking the linkage and how to heal the systme once the main woerktree is moved represent a challenge that involve some hours of debugging, logs, investigation and testing to make it properly set up the initial state of the system and continue with the worktree creation.

Git staging workflow & pristine version control engineering

Context of the problem

Now that the system is getting to its final shape one of the most important tools it should integrate is git, but how to do it properly to ensure a pristine commit history to understand the project, its progression, what has been implemented and how the project as evolved over time.
Also easily look backwards for quickly identification of the implementation details we are looking for. Nor in a messy uncontextualized way but rather having clear context on what are those changes related to, which requisite is being implemented, refactor or updated
Managing docs and code within the same repository carries the risk of “micro commit bloat” only small work and temporary struggle that doesn’t carry a significant weight when backward looking in the future fill the commit history. When reviewing you don’t what to see hundreds of micro commits that doesn’t actually inform of a complete feature.
I’ve analize the possible architectural decision including:

• Workspace squashing and isolated staging of wip commits
• Git worktrees
• Clean commit guidelines
• Clean Room Extraction:  2 branches one for the workspace, containing all the wip messy commits.

Edge cases

But the one that must be picked is the one that adresses the edge cases:

• Probable cross device synchronization commits that should not be part of the final commit history.
• Another key edge case is the synchronization of the cloud repository and the conflict it might cause with the overall sync and the atomic commit history.
• Handling two REQs that modify the same file in different lines
• Complete vision of the doc files from the markdown renderer

The decision

The final decision combines some of the previously mention alternatives and discards others but to make it short the solution is to use worktrees, one per requirement implementation and then clean it out, but this also comes with the frictions of having to update the symlink connection to the markdown editor of preference.
Visually explained the architecture is in a digrama below

Benefits

Semantic commits
The repository becomes self-documenting. Generating >“release notes”, “architectural change log” or revising >the project evolution becomes a mathematical query:

git log --grep="^type"

Structural Bracket Identifier
This makes it possible to audit the overall evolution of a >specific requirement implementation or architectural >decision over the project life by using git log --grep="[ID]". Ease the debugging process and version control >navigation.

Git-Tree
Concurrent file work history collapse solution: by using >the git-tree the edge case of losing track of multi line >edits that belong to different REQs in the same file is >handled gracefully by the work tree capabilities
Dashboard blindness solution: Eliminates dashboard >blindness by using the symlink connection

Workflow & the custom Script

Finding paths to create symlinks, moving trees to a parent folder so the development can began, the squashing of the commits for a clean history and the merge of wortrees to the main branch migth represent minutes of administrative friction. So the implementation of a script that ease the friction parts of the workflow to ensure the developer never gets out of flow due to administrative stuff is a foundational add on to the system.

Automatic Link Connections Refactor Engine

This engine ensures there is no need for manually refactoring of links that connect the related files across the project. Simply imagining updating hundreds of broken links when a name or folder is change, that is not acceptable.

Also helps reducing the friction when creating connections like thinking the relative path to the file.

This way is avoid the debt of having broken links across the traceability system and friction of connecting files is reduce to 0.

Implementation Details

File mapping
Engineer FileSelectionHelper.js which uses a recursive algorithm that serves as a helper method for data collection using node for file path extraction that works as follows:

• Extracts the root folder directories and all files paths and names storing them in a data structure.
• Stores them separately creating a distinction between files and folders.
• Recursively calls itself to scan the directories stored but before deletes the previously scan directory to ensure the algorithm doesn’t fall in an infinite loop.
• Stops once there are no more directories to scan.

Connections Mapping
Implement a helper function PathExtractionHelper.js that will take the found files by FileSelectionHelper.js build a map of the artifacts and the files that mention a connection to them.
Algorithm behavior:

• Loop over all the files of the project
• Reads the file data an check for a connection mention @trace REQ-001 @
• If that artifact exist as a key in the data structure it will append the file, and ensure the already stored connections are not lost
• Otherwise it will append the first file and create the key

Traceability Pipeline Class
To ensure clean code, concern separation and functionality encapsulation I develop a class that contains methods that make possible the traceability engine.
Methods capabilities:

• Find artifact path in the project folder
• Classify artifact connections in different areas
• Build a markdown table with those connections classification and its relative path to the current artifact ready to navigate by a click from any markdown rendering engine (this ensures the builder can easily open and navigate through the artifact related files, probably implementing what the artifact states)
• Write the markdown table to a specific artifact.md file

Engine execution
Once the separated functionalities are encapsulated in its methods and helpers the engine must be orchestrated to provided the expected behavior for this the AutoLinkRefactorExecutionThread.js ensures the execution works as intended leveraging the tools available to ensure the acceptance criteria expected outcome is fulfilled.

Testing
To ensure system integrity this implementation includes 2 test suits:

• 6 unit tests for the standalone methods and helpers
• 1 integration test that ensures that the orchestration of the standalone pieces provide the expected output.

CORE FEATURES

Engineer the requirements structure & artifacts including:

• Kanban Board: How data can be accessed from the requisite Kanban board to increase productivity and maintain a wip (work in progress) of one at the time while visualizing the status and priority of each requisite.
• Artifacts: Create templates for each of the requisite note artifacts: User Stories, acceptance criteria, Analytical breakdown (Operational procedure to guide problem solving ).
• Traceability properties: Define the yaml properties for the REQ artifact to ensure traceability form the kanban and the Maps of content dashboard.

Define a symlink connection usage from the markdown editor to the project folder with the git system

Engineer the architectural decision records structure and artifacts including:

• Kanban Board: How data can be accessed from the ADRs Kanban board visualize decisions that are pending, approved or depracated
• Artifacts: Create templates for each of the requisite note artifacts
• Traceability properties: Define the yaml properties for the ADR artifact to ensure traceability from the kanban and the Maps of content dashboard.

Created a central MOC dashboard that maps automatically ADRs, REQs and Vertical slicing diagrams. Includes also an architectural diagram that gives a big picture of how the system behaves.